BATON: Enhancing Batch-wise Inference Efficiency for Large Language Models via Dynamic Re-batching.

Peizhuang Cong, Qizhi Chen, Haochen Zhao, Tong Yang

Browse the full WWW paper archive.