LongHeads: Multi-Head Attention is Secretly a Long Context Processor.

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Browse the full EMNLP paper archive.