Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis.

Zehua Pei, Hui-Ling Zhen, Lancheng Zou, Xianzhi Yu, Wulong Liu, Sinno Jialin Pan, Mingxuan Yuan, Bei Yu

Browse the full ACL paper archive.