Nexus: Adaptive Upcycling to Efficiently Pretrain Mixture of Experts.

Nikolas Gritsch, Qizhen Zhang, Acyr Locatelli, Sara Hooker, Ahmet stn

Browse the full EMNLP paper archive.