Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling.

Xingyue Huang, Xueying Ding, Mingxuan Ju, Yozen Liu, Neil Shah, Tong Zhao

Browse the full ACL paper archive.