SOA: A Sparsity-Oriented Activation on Sub-layers of FFN of Transformers.

Yulong Meng, Yuan Li, Binhan Chen, Yi Kang

Browse the full ICASSP paper archive.