Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining.

Qi Zhang, Tianqi Du, Haotian Huang, Yifei Wang, Yisen Wang

Browse the full ICML paper archive.