Scrooge: Accelerating Attention Inference in LLMs via Early Termination Mechanism.

Gwangeun Byeon, Seongwook Kim, Taein Kim, Jungmin Lee, Seokin Hong

Browse the full DATE paper archive.