如果你看过《银河系漫游指南》，应该记得地球生物智慧排第一的是海豚（人类只排第三）😂

离别时本来应该是来一句：

“So long! And thanks for all the fish."

但实际情况却是：

Sine Wave Signal

An audio signal, y(t), composed of exactly one sine wave, can be completely described by the parameters $t, A, f$ and $\phi$, $$ y(t) = A \sin(2 \pi f t + \phi) $$ where $t$ represents time in seconds, $A$ is the wave’s amplitude (unit-less), $f$ is its frequency in Hz, and $\phi$ is its phase offset in radians (i.e., where in the cycle the wave is at $t=0$). If $t \ne 0$, then the sine wave appears shifted in time by $\frac{\phi}{2 \pi f}$, where negative values mean “delay” and positive “advance” it.

Fourier Series

Our old pal Fourier told us that any sound can be represented as an infinite summation of sine waves each with their own amplitudes, frequencies, and phase offsets. This means that any sound we hear can be represented as many, many tuples of $t, A, f, \phi$.

An analysis problem, which is equivalent to the DFT:

Given a signal, how to find the amplitude and phase offset of its frequency
components?

A synthesis problem, which is equivalent to the inverse DFT:

Given a set of frequency components and their amplitudes, how can we construct a
signal?

思维实验： 在浅层网络结构的基础上（比如20层），往后面直接添加更多的同映射隐层（identity layers）， 得到的深层网络（比如50层）理论上效果应该不会变差。但实验结果说明，它真会变差。意味着： SGD算法无法找到使得更深层网络性能不变差的参数。

残差网络架构可以解决这个问题。

# Why, What, and How

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions.

训练一个深度（足够深的）神经网络是一件很难的事情（2015年）。 使用“残差”神经网络架构可以更容易地训练足够深的神经网络。 “残差”架构就是把这些中间层作为一个学习输入与输出的残差的函数。

就是说：增加的隐层去学习 $h(x) - x$（残差）而不是 $h(x)$，而输出的是当前隐层的输出加上前 一层的输出 $x$ （同时也是当前层的输入）。