Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages.

Tomasz Limisiewicz, Jir Balhar, David Marecek

Browse the full ACL paper archive.