The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers.

Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar

Browse the full ICLR paper archive.