11-06-2026, 05:05 PM
(11-06-2026, 03:29 PM)bi3mw wrote: You are not allowed to view links. Register or Login to view.While experimenting with the type/token ratio in the VMS corpus, I compared the original curve with a smoothed ideal curve.
You are not allowed to view links. Register or Login to view. (which I believe is a consequence of Zipf's law plus the fact that word counts must be integers) says that the number of distinct words (lexemes, word types) in a text of size N is proportional to sqrt(N). Thus the type/token ratio should be approximately K/sqrt(N). Is that what your "smoothed curve" is?
All the best,--stolfi