A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time.

Yeqi Gao, Zhao Song, Weixin Wang, Junze Yin

Browse the full UAI paper archive.