Samyam Rajbhandari

Publication record assembled from the DBLP archive of ranked conferences.

Papers indexed

Venues

Active years

2013–2026

Best venue rank

Where they publish

Papers

23 indexed papers, newest first.

Year	Venue	Title	Authors
2026	ASPLOS	Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads.	Mert Hidayetoglu, Aurick Qiao, Michael Wyatt, Jeff Rasley, Yuxiong He, Samyam Rajbhandari
2025	EMNLP	SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation.	Aurick Qiao, Zhewei Yao, Samyam Rajbhandari, Yuxiong He
2024	ICLR	ZeRO++: Extremely Efficient Collective Communication for Large Model Training.	Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Xiaoxia Wu, Connor Holmes, Zhewei Yao, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He
2024	PODC	System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models.	Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Reza Yazdani Aminadabi, Shuaiwen Leon Song, Samyam Rajbhandari, Yuxiong He
2023	ICS	A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training.	Siddharth Singh, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He, Abhinav Bhatele
2022	HiPC	1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed.	Conglong Li, Ammar Ahmad Awan, Hanlin Tang, Samyam Rajbhandari, Yuxiong He
2022	ICML	DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale.	Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He
2022	SC	DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale.	Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, Yuxiong He
2021	ICML	1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed.	Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan, Samyam Rajbhandari, Conglong Li, Xiangru Lian, Ji Liu, Ce Zhang, Yuxiong He
2021	SC	ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning.	Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, Yuxiong He
2021	USENIX	ZeRO-Offload: Democratizing Billion-Scale Model Training.	Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, Yuxiong He
2020	KDD	DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters.	Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, Yuxiong He
2020	SC	ZeRO: memory optimizations toward training trillion parameter models.	Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He
2019	ICDM	Fast LSTM Inference by Dynamic Decomposition on Cloud Systems.	Yang You, Yuxiong He, Samyam Rajbhandari, Wenhan Wang, Cho-Jui Hsieh, Kurt Keutzer, James Demmel
2018	ICLR	Learning Intrinsic Sparse Structures within Long Short-Term Memory.	Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li
2018	USENIX	DeepCPU: Serving RNN-based Deep Learning Models 10x Faster.	Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, Yuxiong He
2017	ASPLOS	Optimizing CNNs on Multicores for Scalability, Performance and Goodput.	Samyam Rajbhandari, Yuxiong He, Olatunji Ruwase, Michael Carbin, Trishul M. Chilimbi
2017	PPoPP	Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis.	Samyam Rajbhandari, Fabrice Rastello, Karol Kowalski, Sriram Krishnamoorthy, P. Sadayappan
2016	CC	On fusing recursive traversals of K-d trees.	Samyam Rajbhandari, Jinsung Kim, Sriram Krishnamoorthy, Louis-Nol Pouchet, Fabrice Rastello, Robert J. Harrison, P. Sadayappan
2016	SC	A domain-specific compiler for a parallel multiresolution adaptive numerical simulation environment.	Samyam Rajbhandari, Jinsung Kim, Sriram Krishnamoorthy, Louis-Nol Pouchet, Fabrice Rastello, Robert J. Harrison, P. Sadayappan
2014	ICPP	CAST: Contraction Algorithm for Symmetric Tensors.	Samyam Rajbhandari, Akshay Nikam, Pai-Wei Lai, Kevin Stock, Sriram Krishnamoorthy, P. Sadayappan
2014	SC	A Communication-Optimal Framework for Contracting Distributed Tensors.	Samyam Rajbhandari, Akshay Nikam, Pai-Wei Lai, Kevin Stock, Sriram Krishnamoorthy, P. Sadayappan
2013	SC	A framework for load balancing of tensor contraction expressions via dynamic task partitioning.	Pai-Wei Lai, Kevin Stock, Samyam Rajbhandari, Sriram Krishnamoorthy, P. Sadayappan