PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU.

Yixin Song, Zeyu Mi, Haotong Xie, Haibo Chen

Browse the full SOSP paper archive.