Small and Fast LLMs on Commodity Hardware: Post-Training Quantization in llama. cpp.

Lorenz Sparrenberg, Tobias Deuer, Armin Berger, Rafet Sifa

Browse the full DSAA paper archive.