Position Prediction as an Effective Pretraining Strategy.

Shuangfei Zhai, Navdeep Jaitly, Jason Ramapuram, Dan Busbridge, Tatiana Likhomanenko, Joseph Y. Cheng, Walter Talbott, Chen Huang, Hanlin Goh, Joshua M. Susskind

Browse the full ICML paper archive.