Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality (extended abstract).

Siyu Chen, Heejune Sheen, Tianhao Wang, Zhuoran Yang

Browse the full COLT paper archive.