Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent.

Bo Chen, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song

Browse the full AISTATS paper archive.