How does auto-generating phrases work?

Kai Grossjohann <[email protected]> Thu, 30 Oct 2025 14:49:00 +0100
Newsgroups gmane.comp.jakarta.lucene.user
Message-ID <[email protected]>
--------------3tF4XVCHdzxs6k0Uesqbq8Yj
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

I thought if I have a synonym map that says “synonym alias” is an alias 
for “canonical phrase”, and I noodle “canonical phrase” through the 
query parser, telling it to auto generate multi term queries, I'd get a 
multi term query.  But that doesn't seem to be the case.

The only way to generate multi term queries seems to be when the synonym 
says that “shortsyn” is an alias for “another phrase”, and then noodle 
“shortsyn” through the query parser.  Then I get foo:"another phrase"~1 
which is what I expected.

My use case is as follows: I have some multi-word strings, and I need to 
create queries from them.  And if one of the synonym phrases appears in 
the multi-word string, then I would like to generate a phrase query for 
that part.  For example, given the synonyms mentioned above, if the 
multi-word string is, say, “my synonym alias is nice”, then I'd like to 
generate a query that searches for the word “my”, the phrase “canonical 
phrase”, and the words “is” and “nice”.  Maybe I would like to 
/also/ search for the words “synonym” and “alias”, or the words 
“canonical” and “phrase”, or all four of them, I'm not sure.

This description left out quite a bit of information, I'll paste some 
code below to clarify.

Kai

/**
  * This tests the behavior of the Lucene query
  * builder with synonyms
  */
public class SynonymGraphQueryBuilderTest {

     private static class MyAnalyzer extends Analyzer {
         private final CharArraySet stopwords;
         private final SynonymMap synonyms;

         public MyAnalyzer(Set<String> stopwords, SynonymMap synonyms) {
             this.stopwords = new CharArraySet(stopwords, true);
             this.synonyms = synonyms;
         }

         @Override
         protected TokenStreamComponents createComponents(String 
fieldName) {
             final Tokenizer src = new SimplePatternTokenizer("[a-z0-9]+");
             TokenStream tok = new LowerCaseFilter(src);
             tok = new SynonymGraphFilter(tok, synonyms, true);
             tok = new FlattenGraphFilter(tok);
             tok = new StopFilter(tok, stopwords);
             return new TokenStreamComponents(
                     src::setReader,
                     tok);
         }
     }

     @Test
     void testSynonymPhrases() throws Exception {
         Builder builder = new Builder();

         // canonical phrase <- synonym alias
         CharsRef canonical = Builder.join(new String[] { "canonical", 
"phrase" }, new CharsRefBuilder());
         CharsRef synonym = Builder.join(new String[] { "synonym", 
"alias" }, new CharsRefBuilder());
         builder.add(synonym, canonical, true);

         // another phrase <- shortsyn
         canonical = Builder.join(new String[] { "another", "phrase" }, 
new CharsRefBuilder());
         synonym = Builder.join(new String[] { "shortsyn" }, new 
CharsRefBuilder());
         builder.add(synonym, canonical, true);

         SynonymMap synonyms = builder.build();

         Set<String> stopwords = Set.of("the");

         MyAnalyzer analyzer = new MyAnalyzer(stopwords, synonyms);

         QueryParser queryParser = new QueryParser("foo", analyzer);
         queryParser.setSplitOnWhitespace(true);
         queryParser.setAutoGeneratePhraseQueries(true);
queryParser.setAutoGenerateMultiTermSynonymsPhraseQuery(true);
         queryParser.setPhraseSlop(1);

         Query q = queryParser.parse("canonical phrase");
         assertEquals("foo:canonical foo:phrase", q.toString(),
                 "I was expecting a phrase query here: foo:\"canonical 
phrase\"~1");

         q = queryParser.parse("synonym alias");
         assertEquals("foo:synonym foo:alias", q.toString(),
                 "I was expecting a phrase query here: foo:\"canonical 
phrase\"~1");

         q = queryParser.parse("shortsyn");
         assertEquals("foo:\"another phrase\"~1 foo:shortsyn", q.toString(),
                 "This is what I expected.");

         q = queryParser.parse("another phrase");
         assertEquals("foo:another foo:phrase", q.toString(),
                 "I was expecting a phrase query here: foo:\"another 
phrase\"~1");
     }
}

--------------3tF4XVCHdzxs6k0Uesqbq8Yj--