DjVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving.

Foteini Strati, Sara McAllister, Amar Phanishayee, Jakub Tarnawski, Ana Klimovic

Browse the full ICML paper archive.