Word predictability estimates from language models are not robust to tokenizer vocabulary.

Kien Nguyen, Suhas Arehalli

Browse the full CoNLL paper archive.