PKAS: Predictive KVCache-Aware Scheduling for Faster LLM and Transformer Inferences.

Jie Ye, Avinash Maurya, Krishna Teja Chitty-Venkata, Bogdan Nicolae, Anthony Kougkas, Xian-He Sun

Browse the full HPDC paper archive.