How LLM models try to understand Voynichese

[stopsquark]

Have you tried character-level (or ligature-level, if you want to treat entities 'ch' as one character) tokenization? That is, rather than trying to predict the next word, try to predict the next letter or lettergroup in a line from the previous, and then look at: 

1) the other top-K suggestions in the beam
2) saliency maps
3) which tokens are especially hard or easy to predict- is something like a "ch" often swapped for "sh", for example? 

I've worked on training small LMs for math in the past and found that these sorts of things often give interesting results!