Scaling Laws for Upcycling Mixture-of-Experts Language Models.

Seng Pei Liew, Takuya Kato, Sho Takase

Browse the full ICML paper archive.