(19-02-2026, 03:31 PM)quimqu wrote: You are not allowed to view links. Register or Login to view.Even if entropy is not predictability, they are very linked.
For a well-known measure of predictability of characters separated by distance d, I could modify slightly my very simple code (Java or JavaScript) that calculates the conditional character entropy (H2), to calculate H2(d), using the conditional probability of the character at index + d (modulo the size of the text to avoid border effects), knowing the character at a given index, instead of d = 1 for H2.
Isn't MI(d) = H1 - H2(d) ? Not sure...
H1 = First order character entropy (Shannon's entropy)
H2 = Second order character entropy (conditional entropy)
I would be surprised if H1 - H2(d) doesn't drop to near zero (asymptote at zero) very quickly (approximately when d > 25) and even more quickly (approximately when d > 10) when the text is word-shuffled.