Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization.

Minsu Kim, Seongmin Hong, Ryeowook Ko, Soongyu Choi, Hunjong Lee, Junsoo Kim, Joo-Young Kim, Jongse Park

Browse the full ISCA paper archive.