| 2025 | ICLR | Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks. | Binghui Li, Zhixuan Pan, Kaifeng Lyu, Jian Li |
| 2025 | ICLR | A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules. | Kairong Luo, Haodong Wen, Shengding Hu, Zhenbo Sun, Zhiyuan Liu, Maosong Sun, Kaifeng Lyu, Wenguang Chen |
| 2025 | ICLR | Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias. | Rui Lu, Runzhe Wang, Kaifeng Lyu, Xitai Jiang, Gao Huang, Mengdi Wang |
| 2025 | ICLR | Efficient stagewise pretraining via progressive subnetworks. | Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar |
| 2025 | ICLR | Safety Alignment Should be Made More Than Just a Few Tokens Deep. | Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, Peter Henderson |
| 2025 | ICLR | RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval. | Kaiyue Wen, Xingyu Dang, Kaifeng Lyu |
| 2025 | ICML | Weak-to-Strong Generalization Even in Random Feature Networks, Provably. | Marko Medvedev, Kaifeng Lyu, Dingli Yu, Sanjeev Arora, Zhiyuan Li, Nathan Srebro |
| 2024 | ICLR | A Quadratic Synchronization Rule for Distributed Deep Learning. | Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang |
| 2024 | ICLR | Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking. | Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Shaolei Du, Jason D. Lee, Wei Hu |
| 2024 | ICLR | The Marginal Value of Momentum for Small Learning Rate SGD. | Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li |
| 2024 | ICLR | DistillSpec: Improving Speculative Decoding via Knowledge Distillation. | Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-Franois Kagy, Rishabh Agarwal |
| 2023 | ICLR | Why (and When) does Local SGD Generalize Better than SGD? | Xinran Gu, Kaifeng Lyu, Longbo Huang, Sanjeev Arora |
| 2023 | ICML | Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. | Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee |
| 2021 | ICLR | Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. | Zhiyuan Li, Yuping Luo, Kaifeng Lyu |
| 2020 | ICLR | Gradient Descent Maximizes the Margin of Homogeneous Neural Networks. | Kaifeng Lyu, Jian Li |
| 2019 | ICLR | Theoretical Analysis of Auto Rate-Tuning by Batch Normalization. | Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu |
| 2019 | SODA | Fine-grained Complexity Meets IP = PSPACE. | Lijie Chen, Shafi Goldwasser, Kaifeng Lyu, Guy N. Rothblum, Aviad Rubinstein |
| 2018 | ICALP | Single-Source Bottleneck Path Algorithm Faster than Sorting for Sparse Graphs. | Ran Duan, Kaifeng Lyu, Yuanhang Xie |