MoEfication: Transformer Feed-forward Layers are Mixtures of Experts.

Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Browse the full ACL paper archive.