MoH: Multi-Head Attention as Mixture-of-Head Attention.

Peng Jin, Bo Zhu, Li Yuan, Shuicheng Yan

Browse the full ICML paper archive.