Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition.

Jiyeon Kim, Hyunji Lee, Hyowon Cho, Joel Jang, Hyeonbin Hwang, Seungpil Won, Youbin Ahn, Dohaeng Lee, Minjoon Seo

Browse the full ICLR paper archive.