Spelling-out is not Straightforward: LLMs' Capability of Tokenization from Token to Characters.

Tatsuya Hiraoka, Kentaro Inui

Browse the full EMNLP paper archive.