2 hours ago
(Yesterday, 09:03 AM)nablator wrote: You are not allowed to view links. Register or Login to view.Yes but this is totally irrelevant.
In that post with the two examples I was arguing against the general gibberish hypothesis. Under that hypothesis the terms "spelling reform" (massive or step-by-step) and "language change" are not appropriate. Instead we should discuss changes in the blabbing algorithm and its parameters.
My point was that, if someone in the 1400s decided to create a gibberish text, he very probably would have picked a method that was not stable -- meaning that its parameters would tend to drift over time. That includes the "just write random words" method that G&B used in their experiment, and the T&T "copy and mutate" method.
With the G&B (non)method, the author would not be able to keep all character, digraph, word, and word pair statistics constant over hundreds of pages. As I tried to show with those two sample texts -- which are Italian and German with each line flipped L-R -- he would not have noticed even a change in those statistics as large as that between those two languages. But that is not at all what we see in the VMS.
The T&T method would also tend to drift, because it would be just expanding the statistics of the last few N words and adding the mutation noise. To prevent it from drifting, one must prime it with a seed text with the desired statistics, and fine-tune the mutate procedure so that it pulls the statistics towards those target values.
So it seems that any gibberish theory needs a complicated algorithm A that is fed by a mechanical stable source of random bits. To add a new word, the Author would have to do the equivalent rolling dice or tossing coins to generate a string r of ~8 random bits, and then execute A(wp,r) where wp is the previous word. The algorithm A would have to be at least as complicated as a Markov chain of order 1.
That is one of the reasons why, even beefore my findings of SPS x SBJ, my probability for any gibberish theory was already nearly zero.
Quote:Your claim that the differences between Currier A and B are like those caused by a 'modest' spelling reform is another thing. The orthographic reforms would actually need to be huge, or (more likely) impossible ... and I don't believe it is what we are seeing in the VMS. Your own proposed mapping that would bring the frequencies in the two "languages" closer doesn't describe a plausible orthographic reform, it is too pervasiveThe facts are that the pages of the VMS can be split into two major subsets on the basis of text statistics, "Language A" and "Language B", such that no pages fall into a broad "clean" zone between the two subsets. And every section is in just one of these two sets, except Herbal which is split between them.
Each of these two subsets of pages seem to be divided into separate sub-subsets that seem to correspond to the individual sections. Whether there is more structure it is unclear.
Anyway there are big differences in statistics between the "A" and "B" subsets, and smaller differences between the sections. What can be the causes of these differences?
One cause is topic. We expect that each section, just because it deals with a different topic, will use different words, and the same words with different frequencies. That causes the frequencies of digraphs to be different too.
Is this cause sufficient to explain all differences between sections? For individual sections within each language, maybe yes. But the differences between A and B seem too big for that to be the only cause.
What could cause differences as big as A vs B? A spelling reform, or a change of encryption, or a change of dialect could all do it. They all can produce arbitrarily big changes in the statistics.
Quote:What do you mean? I explicitly allowed that it could be cipher or gibberish.(17-10-2026, 03:24 PM)Jorge_Stolfi wrote: You are not allowed to view links. Register or Login to view.Could then we conclude that the statistical differences between VMS sections, even between Herbal-A and Herbal-B, are more like those caused by a modest spelling/encoding/blabber reform than a change of natural language?This is starting from the (wrong) premise that the only possibilities are 1) a spelling reform 2) different natural languages.
But indeed I do believe, now more than ever, that it is meaningful natural language, not encrypted. You know why...
All the best, --stolfi

