DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction.

Yanqi Zhang, Yuwei Hu, Runyuan Zhao, John C. S. Lui, Haibo Chen

Browse the full SOSP paper archive.