Re: How does auto-generating phrases work?

--000000000000c8831e0642b6b4a4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Kai

Pardon for vide coding, but this sample
https://github.com/mkhludnev/mutlyword-phrase-query-test/blob/3e3f1cce6b2b6=
790970e4a042ddb2967e49d0077/src/test/java/org/example/phrases/MultiWordTest=
s.java#L88


parses plain biword "power grid" without quotes as a bool/should of phrases

org.example.phrases.MultiWordTests#testPhraseQueryGeneratedFromPlainMultiWo=
rdSynonym

Parsed Query for 'power grid': ("electrical grid" "power grid")

Does it look closer to what you are looking for?


On Mon, Nov 3, 2025 at 1:50=E2=80=AFPM Kai Grossjohann <grossjohann@semedy.=
com>
wrote:

> Hi Mikhail,
>
> I tried to change this to false, and this was the result:
>
> java.lang.IllegalArgumentException: setAutoGeneratePhraseQueries(true) is
> disallowed when getSplitOnWhitespace() =3D=3D false
>
> I experimented with other combinations of setSplitOnWhitespace,
> setAutoGeneratePhraseQueries, and
> setAutoGenerateMultiTermSynonymsPhraseQuery.  None of them got me the
> phrase queries I'm looking for.  Though some of them searched for more
> synonyms.
>
> In particular, false/false/true resulted in =E2=80=9Csynonym alias=E2=80=
=9D being parsed
> as Synonym(foo:canonical foo:synonym) Synonym(foo:alias foo:phrase) which
> still doesn't produce the foo:"canonical phrase"~1 that I was looking for=
.
>
> Kai
> On 2025-10-30 18:01, Mikhail Khludnev wrote:
>
> Hello Kaj
>
> Briefly skimming through the letter
>
>          queryParser.setSplitOnWhitespace(true); // shouldn't false be he=
re
> ?
>          queryParser.setAutoGeneratePhraseQueries(true);
> queryParser.setAutoGenerateMultiTermSynonymsPhraseQuery(true);
>          queryParser.setPhraseSlop(1);
>
>          Query q =3D queryParser.parse("canonical phrase");
>          assertEquals("foo:canonical foo:phrase", q.toString(),
>                  "I was expecting a phrase query here: foo:\"canonical
> phrase\"~1");
>
>
>
> On Thu, Oct 30, 2025 at 4:49=E2=80=AFPM Kai Grossjohann<grossjohann@semed=
y.com.invalid> <[email protected]> wrote:
>
>
> I thought if I have a synonym map that says =E2=80=9Csynonym alias=E2=80=
=9D is an alias
> for =E2=80=9Ccanonical phrase=E2=80=9D, and I noodle =E2=80=9Ccanonical p=
hrase=E2=80=9D through the
> query parser, telling it to auto generate multi term queries, I'd get a
> multi term query.  But that doesn't seem to be the case.
>
> The only way to generate multi term queries seems to be when the synonym
> says that =E2=80=9Cshortsyn=E2=80=9D is an alias for =E2=80=9Canother phr=
ase=E2=80=9D, and then noodle
> =E2=80=9Cshortsyn=E2=80=9D through the query parser.  Then I get foo:"ano=
ther phrase"~1
> which is what I expected.
>
> My use case is as follows: I have some multi-word strings, and I need to
> create queries from them.  And if one of the synonym phrases appears in
> the multi-word string, then I would like to generate a phrase query for
> that part.  For example, given the synonyms mentioned above, if the
> multi-word string is, say, =E2=80=9Cmy synonym alias is nice=E2=80=9D, th=
en I'd like to
> generate a query that searches for the word =E2=80=9Cmy=E2=80=9D, the phr=
ase =E2=80=9Ccanonical
> phrase=E2=80=9D, and the words =E2=80=9Cis=E2=80=9D and =E2=80=9Cnice=E2=
=80=9D.  Maybe I would like to
> /also/ search for the words =E2=80=9Csynonym=E2=80=9D and =E2=80=9Calias=
=E2=80=9D, or the words
> =E2=80=9Ccanonical=E2=80=9D and =E2=80=9Cphrase=E2=80=9D, or all four of =
them, I'm not sure.
>
> This description left out quite a bit of information, I'll paste some
> code below to clarify.
>
> Kai
>
> /**
>   * This tests the behavior of the Lucene query
>   * builder with synonyms
>   */
> public class SynonymGraphQueryBuilderTest {
>
>      private static class MyAnalyzer extends Analyzer {
>          private final CharArraySet stopwords;
>          private final SynonymMap synonyms;
>
>          public MyAnalyzer(Set<String> stopwords, SynonymMap synonyms) {
>              this.stopwords =3D new CharArraySet(stopwords, true);
>              this.synonyms =3D synonyms;
>          }
>
>          @Override
>          protected TokenStreamComponents createComponents(String
> fieldName) {
>              final Tokenizer src =3D new SimplePatternTokenizer("[a-z0-9]=
+");
>              TokenStream tok =3D new LowerCaseFilter(src);
>              tok =3D new SynonymGraphFilter(tok, synonyms, true);
>              tok =3D new FlattenGraphFilter(tok);
>              tok =3D new StopFilter(tok, stopwords);
>              return new TokenStreamComponents(
>                      src::setReader,
>                      tok);
>          }
>      }
>
>      @Test
>      void testSynonymPhrases() throws Exception {
>          Builder builder =3D new Builder();
>
>          // canonical phrase <- synonym alias
>          CharsRef canonical =3D Builder.join(new String[] { "canonical",
> "phrase" }, new CharsRefBuilder());
>          CharsRef synonym =3D Builder.join(new String[] { "synonym",
> "alias" }, new CharsRefBuilder());
>          builder.add(synonym, canonical, true);
>
>          // another phrase <- shortsyn
>          canonical =3D Builder.join(new String[] { "another", "phrase" },
> new CharsRefBuilder());
>          synonym =3D Builder.join(new String[] { "shortsyn" }, new
> CharsRefBuilder());
>          builder.add(synonym, canonical, true);
>
>          SynonymMap synonyms =3D builder.build();
>
>          Set<String> stopwords =3D Set.of("the");
>
>          MyAnalyzer analyzer =3D new MyAnalyzer(stopwords, synonyms);
>
>          QueryParser queryParser =3D new QueryParser("foo", analyzer);
>          queryParser.setSplitOnWhitespace(true);
>          queryParser.setAutoGeneratePhraseQueries(true);
> queryParser.setAutoGenerateMultiTermSynonymsPhraseQuery(true);
>          queryParser.setPhraseSlop(1);
>
>          Query q =3D queryParser.parse("canonical phrase");
>          assertEquals("foo:canonical foo:phrase", q.toString(),
>                  "I was expecting a phrase query here: foo:\"canonical
> phrase\"~1");
>
>          q =3D queryParser.parse("synonym alias");
>          assertEquals("foo:synonym foo:alias", q.toString(),
>                  "I was expecting a phrase query here: foo:\"canonical
> phrase\"~1");
>
>          q =3D queryParser.parse("shortsyn");
>          assertEquals("foo:\"another phrase\"~1 foo:shortsyn",
> q.toString(),
>                  "This is what I expected.");
>
>          q =3D queryParser.parse("another phrase");
>          assertEquals("foo:another foo:phrase", q.toString(),
>                  "I was expecting a phrase query here: foo:\"another
> phrase\"~1");
>      }
> }
>
>
>

--=20
Sincerely yours
Mikhail Khludnev

--000000000000c8831e0642b6b4a4--