LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training.

Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, Yu Cheng

Browse the full EMNLP paper archive.