| 2025 | ACL | Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models. | Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin |
| 2025 | ACL | Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images. | Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber |
| 2025 | ICLR | Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View. | Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang, Tengyu Ma |
| 2025 | ICLR | RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval. | Kaiyue Wen, Xingyu Dang, Kaifeng Lyu |
| 2025 | ICLR | From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency. | Kaiyue Wen, Huaqing Zhang, Hongzhou Lin, Jingzhao Zhang |
| 2025 | ICML | Task Generalization with Autoregressive Compositional Structure: Can Learning from D Tasks Generalize to DT Tasks? | Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen, Hongzhou Lin, Jingzhao Zhang, Mikhail Belkin |
| 2025 | ICML | Overtrained Language Models Are Harder to Fine-Tune. | Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan |
| 2023 | ICLR | How Sharpness-Aware Minimization Minimizes Sharpness? | Kaiyue Wen, Tengyu Ma, Zhiyuan Li |
| 2023 | ICLR | Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models. | Kaiyue Wen, Jiaye Teng, Jingzhao Zhang |
| 2022 | EMNLP | Finding Skill Neurons in Pre-trained Transformer-based Language Models. | Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, Juanzi Li |
| 2022 | NAACL | On Transferability of Prompt Tuning for Natural Language Processing. | Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou |