Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models.

Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin

Browse the full ACL paper archive.