PTStore (Prefix Tensor Store): Distributed Prefix Caching and Replication for High Throughput Inference Serving.

Meghana Maghyastha, Robert Underwood, Randal C. Burns, Bogdan Nicolae

Browse the full EuroPar paper archive.