Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA.

Qingyun Jin, Xiaohui Song, Feng Zhou, Zengchang Qin

Browse the full EMNLP paper archive.