Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy.

Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen

Browse the full ICLR paper archive.