Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient.

George Wang, Jesse Hoogland, Stan van Wingerden, Zach Furman, Daniel Murfet

Browse the full ICLR paper archive.