Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function.

Wojciech Tarnowski, Piotr Warchol, Stanislaw Jastrzebski, Jacek Tabor, Maciej A. Nowak

Browse the full AISTATS paper archive.