Re: Per-page statistics

Simon Giesecke <[email protected]> Thu, 25 Dec 2003 15:29:03 +0100
Newsgroups gmane.science.linguistics.wikipedia.international
Message-ID <[email protected]>
> Von: Roozbeh Pournader <roozbeh-mXXj517/[email protected]>
> Datum: 24. Dezember 2003 15:43:36 Uhr MEZ

Dear Roozbeh,

thank you very much for your reply. I'm sorry for my admittedly naive 
view of the problem. I was well aware of your point b), but I thought 
that to get a rough estimate of the comprehensibility, the number of 
words would be enough. I'm not quite sure if your point a) is merely a 
technical problem, but the main point will probably be your point c). 
To measure the comprehensibility by the number of words was just my 
first (bad) guess, which you seem to understand from my viewpoint. Then 
probably a nice idea were to define a language(-family)-specific 
comprehensibility measure, e.g. measuring texts in Germanic languages 
in words with a language-specific correctional factor. If you say, such 
a measure would not be adequate for Persian texts, there is still the 
possibility that another automatically calculable measure exists? It 
should still give only a rough estimate.

Kind regards
Simon

> On Wed, 2003-12-24 at 17:57, Simon Giesecke wrote:
>> Other languages: Dansk (400 words), Deutsch (8000 words), Nederlands
>> (50 words)
>
> a) Computing the number of words in the Unicode era is not easy. For
> example, Thai doesn't use a visible interword space, and CJK concept of
> words and spaces are a little different.
>
> b) Sometimes a single German word contains information equivalent to
> three English words. Putting them next to each other will suggest 
> people
> to compare them, which is just idiotic.
>
> c) Being a Persian speaker myself, I've never heard about counting the
> size of Persian articles or novels in words (although that's a common
> concept in Europe or North America). A Persian reader (even very
> educated ones) don't have any clue about how much information 1000
> Persian words may contain.
>
> roozbeh