Re: Native query language

Hi Tom:

In message <SJ0PR08MB79488ADEC4C4E18C3BA7543DCA599@SJ0PR08MB7948.namprd08.prod.
outlook.com>,
Tom Ekberg writes:
>I have a user that wanted to search issues that have "NO LINE" and
>autofax. I thought we were using xapien so I entered "no line" AND
>autofax in the 'all text' part of the search page.

Using native (and possibly others), AND (any capitalization) is a stop
word (see backends/indexer_common.py) and it is ignored.  IIRC native
is always an AND search, and there is no phrasing (quoted string)
support.

So it should return the same as:

  no line autofax

basically it does a (pseudo SQL):

   select messgeid from _words where word == 'no'
   intersect
   select messgeid from _words where word == 'line'
   intersect
   select messgeid from _words where word == autofax

It is a message based intersect not an issue based intersection.

See: https://issues.roundup-tracker.org/issue1155657

>It returned 30 issues where only 1 had both parts of the search and
>missed another matching issue.

That's unexpected. Is it possible the missed issue has all the words
spread across different messages? If so than it's undesirable but
expected.

>It turns out that neither xapian nor woosh are installed, so,
>according to the config.ini the native search is used.
>
>I searched the doc/ directory and the "Roundup User Guide" but didn't
>find much on the native query language, with regard to 'all text'.

There is no "query language" per se. It's an AND token lookup per
unit.  The unit IIRC is message, title or other string field with
indexme=yes set.

> [... search resorting to grep of messages elided]

>This query identified the 9 issues that contained "no line". It was a
>simple process to see which issues in that list also contain autofax
>using the issue search page.
>
>There must be an easier way.

Whoosh does have a query language.

   https://issues.roundup-tracker.org/issue2551188

but we don't use it. I am not sure what xapian has.

In theory sqlite, postgres and mysql all have full text search support
(which can include substring/trigram searching) but there is no code
that uses them.

Do you have a suggestion on where to put further documentation on how
full text search works in native mode? Which doc would you expect to
cover that?

Sorry I don't have better news.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.