Re: Suggestion: 'aci'(<-'at')
Martin Porter <[email protected]> Tue, 1 Apr 2014 14:37:24 +0100
| Newsgroups | gmane.comp.search.snowball |
|---|---|
| Message-ID | <CALc6duw3Yr4pCqzQXUAN4vtrZg1He5-xCqEJ8R+UXgqVyR_TtQ@mail.gmail.com> |
Hi Chris, thanks for your interest here. It is going back quite a while now, but I think -acy was considered for inclusion, but finally not admitted. You are quite right in saying it should be limited to R2, but even then, the effect of its removal is not so very striking. (I've tended to follow the idea that if removing a suffix helps 60% of the time, but in the other 40% doesn't help or harms, not to include it -- you really want 80 to 90% improvement. And also not to worry too much about rare instances. I realise these assumptions can be challenged of course.) In the vocabulary used with the English stemmer, you find these instances, v accuracy v advocacy n aristocracy ? conspiracy - curacy v delicacy n democracy n diplomacy ? efficacy - fallacy ? inefficacy v intimacy - legacy v legitimacy - lunacy v obduracy v obstinacy - privacy v profligacy ? supremacy R1 cases are maked '-'. When -acy corresponds to an -at (noun) form it is marked 'n', when to an -ate (verb or adjective) form it is marked 'v'. Residual forms are marked '?'. It is the v-cases that lead to proper conflation, or 8 out of 20 (40%). Actually, it might be included, but it is, I think, a borderline case. In the case of -arism' endings, I think you are quite correct in saying that it should stem to -ari. But it's important to recognise how rare the ending is. In the snowball test vocabulary there are actually no cases. In the YAWL wordlist (my copy of it is a bit old), there are 7 instances in a vocabulary of over a quarter of a million words. Here they are: antimilitarism hyperpituitarism hypopituitarism militarism monetarism parliamentarism voluntarism Three of these you instance. (When I see things like 'hyperpituitarism' I always think of that line from Clueless "try and use it in a sentence today" -- https://www.youtube.com/watch?v=lOS04eZ0Xmo -- end of silly joke.) In short, I think suffix inclusion must be judged against percentage improvement and suffix rarity. I had thought of collecting together, at least for English, suggestions made over the years for suffix inclusion and exception words. These ideas are always interesting. This could make an extra page of annotations on the snowball site. Martin