The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws.

Tian Jin, Ahmed Imtiaz Humayun, Utku Evci, Suvinay Subramanian, Amir Yazdanbakhsh, Dan Alistarh, Gintare Karolina Dziugaite

Browse the full ICLR paper archive.