The following warnings occurred:
Warning [2] Undefined variable $can_link - Line: 6 - File: showthread.php(1658) : eval()'d code PHP 8.5.12 (Linux)
File Line Function
/inc/class_error.php 157 errorHandler->error
/showthread.php(1658) : eval()'d code 6 errorHandler->error_callback
/showthread.php 1658 eval




Liaisons in the Voynich text
#29
(06-11-2026, 09:09 PM)quimqu wrote: You are not allowed to view links. Register or Login to view.I put the prohibited pairs in a table, and I marked the ones that [need repair]

Good idea!

I would add more generally 
  • any token with two or more gallows must be split somewhere between them.
  • q<any> where <any> is not o should be repaired to qo, by inserting or replacing the <any>
  • ir should be repaired to iin
  • m should be expanded to iin
  • Ih should be repaired to Ch
  • Rh should be repaired to Sh.
  • ITh, IKh, IPh, IFh should be repaired to CTh, CKh.CPh, CHh
  • any word with n not at the end should be split after the n.
  • hh should be repaired to he
  • any token that ends in n, in, iin, or iiin and not preceded by a or o should be repaired by inserting a before it.
  • that done, any i that is not [preceded by i,  a or o and followed by i or n] should be changed to e if the preceding glyph is h, k, t, p, f, or e (repeat this fix as needed). 

More rules can be devised.  Like in any spelling corrector, there will be a tension between how many errors are fixed and how many correct tokens are broken.

Here is a more general method that could give better results and requires fewer explicit choices.  Choose a word structure model that will define what is a 'valid' word.  (I propose mine, stay tuned.) Get the vocabulary V (set of all word types that occur) and compute the frequency F(v) of each.  

Then take each lexeme (word type) v and enumerate the set X(v) all the strings x that are a few edit steps away from v.  For each of these, let P(x|v) be a probability that v has mutated to x because of error at some step.  This is (say) 80% if x = v, then decreases according to the number and type of the edit steps needed to turn v into x.  The probabilities P(x|v) summed over X(v) must be 1. 

Then, for each u,v in V, use Bayes's formula with probabilities F(v) and P(u|v) to estimate the probability P(v|u) that u was actually some other word v that mutated to u.

Hopefully for most words that occur at least a few times we will find that P(v|u) is definitely maximum when v=u; that is, the word u should not be corrected.  If the maximum is some v different from u, then we should consider correcting u to v.

Makes sense?

All the best, --stolfi
Reply


Messages In This Thread
Liaisons in the Voynich text - by quimqu - 03-11-2026, 10:06 AM
RE: Liaisons in the Voynich text - by nablator - 03-11-2026, 10:48 AM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 11:07 AM
RE: Liaisons in the Voynich text - by ololololo - 03-11-2026, 11:25 AM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 11:45 AM
RE: Liaisons in the Voynich text - by Bluetoes101 - 03-11-2026, 12:20 PM
RE: Liaisons in the Voynich text - by Bernd - 03-11-2026, 12:27 PM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 12:58 PM
RE: Liaisons in the Voynich text - by tavie - 03-11-2026, 02:34 PM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 03:45 PM
RE: Liaisons in the Voynich text - by nablator - 03-11-2026, 03:15 PM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 03:52 PM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 04:36 PM
RE: Liaisons in the Voynich text - by nablator - 03-11-2026, 05:35 PM
RE: Liaisons in the Voynich text - by tavie - 03-11-2026, 10:24 PM
RE: Liaisons in the Voynich text - by quimqu - 03-11-2026, 11:16 PM
RE: Liaisons in the Voynich text - by todihk - 04-11-2026, 12:15 PM
RE: Liaisons in the Voynich text - by ololololo - 04-11-2026, 12:59 PM
RE: Liaisons in the Voynich text - by quimqu - 04-11-2026, 09:28 PM
RE: Liaisons in the Voynich text - by todihk - 05-11-2026, 10:03 AM
RE: Liaisons in the Voynich text - by quimqu - 05-11-2026, 10:13 AM
RE: Liaisons in the Voynich text - by tavie - 04-11-2026, 11:16 PM
RE: Liaisons in the Voynich text - by quimqu - 04-11-2026, 11:33 PM
RE: Liaisons in the Voynich text - by JoJo_Jost - 05-11-2026, 06:41 AM
RE: Liaisons in the Voynich text - by quimqu - 05-11-2026, 10:32 AM
RE: Liaisons in the Voynich text - by Torsten - 05-11-2026, 02:26 PM
RE: Liaisons in the Voynich text - by quimqu - 06-11-2026, 09:09 PM
RE: Liaisons in the Voynich text - by Jorge_Stolfi - 06-11-2026, 11:08 PM
RE: Liaisons in the Voynich text - by nablator - 07-11-2026, 03:55 PM
RE: Liaisons in the Voynich text - by tavie - 06-11-2026, 09:25 PM
RE: Liaisons in the Voynich text - by todihk - 07-11-2026, 08:33 PM
RE: Liaisons in the Voynich text - by nablator - 07-11-2026, 09:37 PM
RE: Liaisons in the Voynich text - by quimqu - 07-11-2026, 10:42 PM
RE: Liaisons in the Voynich text - by quimqu - 07-11-2026, 11:15 PM
RE: Liaisons in the Voynich text - by nablator - 07-11-2026, 11:30 PM
RE: Liaisons in the Voynich text - by quimqu - 07-11-2026, 11:59 PM
RE: Liaisons in the Voynich text - by nablator - 08-11-2026, 12:08 AM
RE: Liaisons in the Voynich text - by quimqu - 08-11-2026, 11:31 AM
RE: Liaisons in the Voynich text - by todihk - 08-11-2026, 01:07 AM
RE: Liaisons in the Voynich text - by Mauro - 08-11-2026, 01:08 PM
RE: Liaisons in the Voynich text - by ololololo - 08-11-2026, 11:50 AM
RE: Liaisons in the Voynich text - by hbaraiki - Yesterday, 03:46 PM

Forum Jump:


Users browsing this thread: 4 Guest(s)