“The Library of Babel” by Jorge Luis Borges (1941)

[Mauro]

Another interesting detail is: "[a book] made up of the letters MCV, perversely repeated from the first line to the last" - the most dramatic example of a hopelessly meaningless text that has character conditional entropy equal to zero.

To be pedantic, the average character entropy would be about 20/N bits, where N is the length of the book.  The 20 bits is a guess at the size of a program that would print out that text, on the Librar'y's Official Turing Machine.

All the best, --stolfi

Should'nt Shannon 1st-order entropy be -3*(1/3*ln(1/3)) =~ 1.099, independent from the length of the book?

The Shannon entropy of a stream of characters is how many bits you need, on average, to specify the next character.   IIRC, in Borges story the alphabet was only the 26 capital Latin letters.  It seems reasonable to assume that each of these letters can occur at each position in any book with the same probability, independently of the rest of the text in that book.  Then the Shannon entropy per character (for any order k), will be log_2(26) = ln(26)/ln(2) = ~4.700 bits.

I agree, you were talking about a generic text (with an uniform distribution of N characters). My value of 1.099 was for the specific text mentioned by MarcoP (a continuos repeat of the characters MCV, N=3). [ps.: I use the symbol 'ln' for 'logarithm in base 2']