The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval.

Ting-Rui Chiang, Dani Yogatama

Browse the full ACL paper archive.