Re: Accents, endings and chaining

According to Lachlan Andrew:
> Greetings Gilles,
> 
> As I understand Dominique's problem, it is not an issue of chaining 
> the "accents" fuzzy rule with the "endings" fuzzy rule.  He is 
> searching for a word with an accent.  If it matches a word in the 
> database with an accent, then it should not need a fuzzy algorithm to 
> register a match.  I thought that "accents" causes "acouphène" to 
> match "acouphene" (without the accend).

I think you're right.  I likely jumped to the wrong conclusion, as it
resembles a problem that's come up many times before.  More likely, the
problem is that "herbe" exists in Dominique's francais.0 dictionary, or
whatever he used to generate the endings database, but "acouphène" isn't.
He'd need to add it, with the appropriates suffixes for pluralization
(usually "/S"), and then do "htfuzzy endings" again.

The big problem with the current endings algorithm is its dependence on
a static dictionary which may be incomplete.  The stemming algorithms
which Neal talked about, and wants to add to ht://Dig, adapt automatically
to whatever words get indexed, based on a set of rules for stemming words.

> Dominique, could you confirm that both the document and the query have 
> an accent?  If that is the case, then we *may* be able to fix this 
> problem without needing to chain fuzzy rules.

Also, please check the dictionary file used by your endings_dictionary
attribute in htdig.conf, to make sure it contains the proper pluralization
rules for any words for which you have a problem, as for acouphène in
this case.

> On Thu, 3 Jun 2004 02:07 am, Gilles Detillieux wrote:
> > Dominique had written:
> > > > 3- I have a problem with the accent and plurials.
> > > >
> > > > If a search for "herbe" or "herbes", no problems.  But, if the
> > > > works have an accent, like "acouphène", htdig have a problem to
> > > > find the plural.
> > > >
> > > > herbe: 136 results
> > > > herbes: 136 results
> > > >
> > > > acouphène:   6 results
> > > > acouphènes:  25 results
> > > >
> > > >
> > > > search_algorithm:       exact:1 endings:1 prefix:1 accent:1
> > > > synonyms:0,5
> >
> > htsearch does not yet support chaining of fuzzy match algorithms,
> > so the results of the accents algorithms don't have the endings
> > algorithm applied to them (nor vice-versa).


-- 
Gilles R. Detillieux              E-mail: <[email protected]>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)


-------------------------------------------------------
This SF.Net email is sponsored by: GNOME Foundation
Hackers Unite!  GUADEC: The world's #1 Open Source Desktop Event.
GNOME Users and Developers European Conference, 28-30th June in Norway
http://2004/guadec.org
_______________________________________________
ht://Dig Developer mailing list:
[email protected]
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-dev