SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices.

Xiangwen Zhuge, Shen Xu, Fan Dang, Xuan Ding, Danyang Li, Tianxiang Hao, Qiang Ma, Xin Miao, Yahui Han, Zheng Yang

Browse the full IWQoS paper archive.