Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings.

Chunsheng Zuo, Pavel Guerzhoy, Michael Guerzhoy

Browse the full COLING paper archive.