Re: Removing été from French stop words
Philippe Ouellet <[email protected]> Wed, 15 Apr 2020 20:14:55 -0400
| Newsgroups | gmane.comp.search.snowball |
|---|---|
| Message-ID | <[email protected]> |
--===============2343962547442029683== Content-Type: multipart/alternative; boundary="Apple-Mail=_D1EED0EF-716D-4063-9D87-E22028D6EAD9" --Apple-Mail=_D1EED0EF-716D-4063-9D87-E22028D6EAD9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Could you give me the link to the file in the repo? I have no idea where = that is. I only did notice =C3=A9t=C3=A9, because one of our product name = contains that word, but you are right about aura and avions.=20 Do we need to remove avions from the stop word, if it get changed to its = singular form during analysis? I am having second thought: removing =C3=A9t=C3=A9 could have a great = impact on the search result, someone searching for =E2=80=9Csummer=E2=80=9D= would result in finding all results containing the past tense form of = =E2=80=9Cto be=E2=80=9D: the impact is huge. Is there a way to make =E2=80=9Ca =C3=A9t=C3=A9=E2=80=9D the stop word = instead? -- Philippe Ouellet Web Developer https://camellia-sinensis.com > On Apr 15, 2020, at 20:05, Olly Betts <[email protected]> wrote: >=20 > On Wed, Apr 15, 2020 at 05:44:35PM -0400, Philippe Ouellet wrote: >> I propose to remove =E2=80=9C=C3=A9t=C3=A9=E2=80=9D and =E2=80=9C=C3=A9= t=C3=A9s=E2=80=9D from the French stop words. It is >> true that they are a form for the verb =E2=80=9Cto be=E2=80=9D, but = it also mean >> =E2=80=9Csummer=E2=80=9D, which should not be a stop word. >=20 > Sounds good to me. It looks like there are other entries with the = same > problem - e.g. "aura" and "avions" are both also nouns. >=20 > But my French is rudimentary at best. Could you review the whole list > and open a PR against the snowball-website repo with your proposed > changes? >=20 > I'd suggest we comment out such entries ("|" is the comment character > here) with a note as to why they are omitted. That should help avoid > future requests to add the apparently missing entries. >=20 > Cheers, > Olly --Apple-Mail=_D1EED0EF-716D-4063-9D87-E22028D6EAD9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">Could= you give me the link to the file in the repo? I have no idea where that = is.<div class=3D""><br class=3D""></div><div class=3D"">I only did = notice =C3=A9t=C3=A9, because one of our product name contains that = word, but you are right about aura and avions. </div><div = class=3D""><br class=3D""></div><div class=3D"">Do we need to remove = avions from the stop word, if it get changed to its singular form during = analysis?</div><div class=3D""><br class=3D""></div><div class=3D"">I am = having second thought: removing =C3=A9t=C3=A9 could have a great impact = on the search result, someone searching for =E2=80=9Csummer=E2=80=9D = would result in finding all results containing the past tense form of = =E2=80=9Cto be=E2=80=9D: the impact is huge.</div><div class=3D""><br = class=3D""></div><div class=3D"">Is there a way to make =E2=80=9Ca = =C3=A9t=C3=A9=E2=80=9D the stop word instead?</div><div class=3D""><br = class=3D""><div class=3D""> <div dir=3D"auto" style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, = 0); letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: = break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div>--<br class=3D"">Philippe Ouellet<br class=3D"">Web = Developer</div><div><a href=3D"https://camellia-sinensis.com" = class=3D"">https://camellia-sinensis.com</a></div></div> </div> <div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On Apr 15, 2020, at 20:05, Olly Betts <<a = href=3D"mailto:[email protected]" class=3D"">[email protected]</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"">On Wed, Apr 15, 2020 at 05:44:35PM -0400, Philippe Ouellet = wrote:<br class=3D""><blockquote type=3D"cite" class=3D"">I propose to = remove =E2=80=9C=C3=A9t=C3=A9=E2=80=9D and =E2=80=9C=C3=A9t=C3=A9s=E2=80=9D= from the French stop words. It is<br class=3D"">true that they are a = form for the verb =E2=80=9Cto be=E2=80=9D, but it also mean<br = class=3D"">=E2=80=9Csummer=E2=80=9D, which should not be a stop word.<br = class=3D""></blockquote><br class=3D"">Sounds good to me. It looks = like there are other entries with the same<br class=3D"">problem - e.g. = "aura" and "avions" are both also nouns.<br class=3D""><br = class=3D"">But my French is rudimentary at best. Could you review = the whole list<br class=3D"">and open a PR against the snowball-website = repo with your proposed<br class=3D"">changes?<br class=3D""><br = class=3D"">I'd suggest we comment out such entries ("|" is the comment = character<br class=3D"">here) with a note as to why they are omitted. = That should help avoid<br class=3D"">future requests to add the = apparently missing entries.<br class=3D""><br class=3D"">Cheers,<br = class=3D""> Olly<br = class=3D""></div></div></blockquote></div><br = class=3D""></div></body></html>= --Apple-Mail=_D1EED0EF-716D-4063-9D87-E22028D6EAD9-- --===============2343962547442029683== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Snowball-discuss mailing list [email protected] https://lists.tartarus.org/mailman/listinfo/snowball-discuss --===============2343962547442029683==--