"The Currier languages revisited" revisited

[Jorge_Stolfi]

there is no continuous Bayesian likelihood "A-like" or "B-like" value, there's binary "in the clump on the left" or "in the clump on the right."

Well, that is a rather binary way to put it.     

Another way to describe is is that the pages of each section (counting Herbal-A and herbal-B as two sections) comprise a broad cloud of points. We can approximate each of these clouds by an N-dimensional Gaussian distribution, each with its own mean, eigenvectors, and eigenvalues.  Then we find that, when projected along the main overall PCA axes, some of the clouds have substantial overlap, while some are well separated by several standard deviations.  Specifically, that there are two main classes of sections, A and B, such that each A cloud is well-separated from every B cloud.

That is important information, but one should also note that the clouds in each set are clearly distinct as well -- just not as distinct as the A-clouds from B-clouds.  Thus the A-B "language" distinction is only a matter of degree; not necessarily a "fundamentally different difference".  Not necessarily more significant than the Bio-Starred difference.

Moreover, as I explained before, if two clouds are well-separated, that will not necessarily show up in the PCA projection.  Even if the PCA is run on those two clouds only.   PCA finds the directions along which the set of all points has maximum spread; not the directions along which it can be split into separate clusters.

Thus the question remains of how well separated are (say) Bio and Starred, or Pharma and Herbal-A.

Another point that should be stressed is that one should use statistics of words, not letters and digraphs.  The frequency of a letter or digraph is determined by the common words that contain it.  Thus letter and digraph statistics are arbitrary projections of word statistics.  Analyzing languages by their digraph statistics is like studying zoology by considering only the colors of the animals' fur.  Sure, it will show that zebras are very different  from black bears and ravens, but...

All the best, --stolfi