Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned.

Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, Ivan Titov

Browse the full ACL paper archive.