Residual Network

A deep neural network architecture introduced by He et al. in 2015 that adds shortcut connections allowing the output of one layer to bypass several subsequent layers and be added directly to a later layer's output. ResNets solved the degradation problem that made very deep networks hard to train, enabling architectures with hundreds of layers and winning multiple image recognition benchmarks in 2015. Residual connections have since become a standard component in transformers and most modern deep learning architectures.