Re: How does auto-generating phrases work?

--0000000000008d04c2064263341c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Kaj

Briefly skimming through the letter

         queryParser.setSplitOnWhitespace(true); // shouldn't false be here
?
         queryParser.setAutoGeneratePhraseQueries(true);
queryParser.setAutoGenerateMultiTermSynonymsPhraseQuery(true);
         queryParser.setPhraseSlop(1);

         Query q =3D queryParser.parse("canonical phrase");
         assertEquals("foo:canonical foo:phrase", q.toString(),
                 "I was expecting a phrase query here: foo:\"canonical
phrase\"~1");



On Thu, Oct 30, 2025 at 4:49=E2=80=AFPM Kai Grossjohann
<[email protected]> wrote:

> I thought if I have a synonym map that says =E2=80=9Csynonym alias=E2=80=
=9D is an alias
> for =E2=80=9Ccanonical phrase=E2=80=9D, and I noodle =E2=80=9Ccanonical p=
hrase=E2=80=9D through the
> query parser, telling it to auto generate multi term queries, I'd get a
> multi term query.  But that doesn't seem to be the case.
>
> The only way to generate multi term queries seems to be when the synonym
> says that =E2=80=9Cshortsyn=E2=80=9D is an alias for =E2=80=9Canother phr=
ase=E2=80=9D, and then noodle
> =E2=80=9Cshortsyn=E2=80=9D through the query parser.  Then I get foo:"ano=
ther phrase"~1
> which is what I expected.
>
> My use case is as follows: I have some multi-word strings, and I need to
> create queries from them.  And if one of the synonym phrases appears in
> the multi-word string, then I would like to generate a phrase query for
> that part.  For example, given the synonyms mentioned above, if the
> multi-word string is, say, =E2=80=9Cmy synonym alias is nice=E2=80=9D, th=
en I'd like to
> generate a query that searches for the word =E2=80=9Cmy=E2=80=9D, the phr=
ase =E2=80=9Ccanonical
> phrase=E2=80=9D, and the words =E2=80=9Cis=E2=80=9D and =E2=80=9Cnice=E2=
=80=9D.  Maybe I would like to
> /also/ search for the words =E2=80=9Csynonym=E2=80=9D and =E2=80=9Calias=
=E2=80=9D, or the words
> =E2=80=9Ccanonical=E2=80=9D and =E2=80=9Cphrase=E2=80=9D, or all four of =
them, I'm not sure.
>
> This description left out quite a bit of information, I'll paste some
> code below to clarify.
>
> Kai
>
> /**
>   * This tests the behavior of the Lucene query
>   * builder with synonyms
>   */
> public class SynonymGraphQueryBuilderTest {
>
>      private static class MyAnalyzer extends Analyzer {
>          private final CharArraySet stopwords;
>          private final SynonymMap synonyms;
>
>          public MyAnalyzer(Set<String> stopwords, SynonymMap synonyms) {
>              this.stopwords =3D new CharArraySet(stopwords, true);
>              this.synonyms =3D synonyms;
>          }
>
>          @Override
>          protected TokenStreamComponents createComponents(String
> fieldName) {
>              final Tokenizer src =3D new SimplePatternTokenizer("[a-z0-9]=
+");
>              TokenStream tok =3D new LowerCaseFilter(src);
>              tok =3D new SynonymGraphFilter(tok, synonyms, true);
>              tok =3D new FlattenGraphFilter(tok);
>              tok =3D new StopFilter(tok, stopwords);
>              return new TokenStreamComponents(
>                      src::setReader,
>                      tok);
>          }
>      }
>
>      @Test
>      void testSynonymPhrases() throws Exception {
>          Builder builder =3D new Builder();
>
>          // canonical phrase <- synonym alias
>          CharsRef canonical =3D Builder.join(new String[] { "canonical",
> "phrase" }, new CharsRefBuilder());
>          CharsRef synonym =3D Builder.join(new String[] { "synonym",
> "alias" }, new CharsRefBuilder());
>          builder.add(synonym, canonical, true);
>
>          // another phrase <- shortsyn
>          canonical =3D Builder.join(new String[] { "another", "phrase" },
> new CharsRefBuilder());
>          synonym =3D Builder.join(new String[] { "shortsyn" }, new
> CharsRefBuilder());
>          builder.add(synonym, canonical, true);
>
>          SynonymMap synonyms =3D builder.build();
>
>          Set<String> stopwords =3D Set.of("the");
>
>          MyAnalyzer analyzer =3D new MyAnalyzer(stopwords, synonyms);
>
>          QueryParser queryParser =3D new QueryParser("foo", analyzer);
>          queryParser.setSplitOnWhitespace(true);
>          queryParser.setAutoGeneratePhraseQueries(true);
> queryParser.setAutoGenerateMultiTermSynonymsPhraseQuery(true);
>          queryParser.setPhraseSlop(1);
>
>          Query q =3D queryParser.parse("canonical phrase");
>          assertEquals("foo:canonical foo:phrase", q.toString(),
>                  "I was expecting a phrase query here: foo:\"canonical
> phrase\"~1");
>
>          q =3D queryParser.parse("synonym alias");
>          assertEquals("foo:synonym foo:alias", q.toString(),
>                  "I was expecting a phrase query here: foo:\"canonical
> phrase\"~1");
>
>          q =3D queryParser.parse("shortsyn");
>          assertEquals("foo:\"another phrase\"~1 foo:shortsyn",
> q.toString(),
>                  "This is what I expected.");
>
>          q =3D queryParser.parse("another phrase");
>          assertEquals("foo:another foo:phrase", q.toString(),
>                  "I was expecting a phrase query here: foo:\"another
> phrase\"~1");
>      }
> }
>


--=20
Sincerely yours
Mikhail Khludnev

--0000000000008d04c2064263341c--