Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer.

Boan Liu, Liang Ding, Li Shen, Keqin Peng, Yu Cao, Dazhao Cheng, Dacheng Tao

Browse the full ECAI paper archive.