Re: Native query language

 > Do you have a suggestion on where to put further documentation on how full text search works in native mode? Which doc would you expect to cover that?

The file doc/overview.txt has a section titled "Browsing and Searching" at the end of the file. The "Roundup User Guide" has a section titled "Searching Page". Either of these would be a place to document full text searching using native mode.

Tom Ekberg
Senior Computer Specialist
Department of Laboratory Medicine and Pathology
4th Floor, Pat Steel Building, currently WFH
Home: (253) 561-2509
Email: [email protected]


-----Original Message-----
From: [email protected] <[email protected]> 
Sent: Wednesday, January 19, 2022 11:13 AM
To: Tom Ekberg <[email protected]>
Cc: Roundup Users ([email protected]) <[email protected]>
Subject: Re: [Roundup-users] Native query language

Hi Tom:

In message <SJ0PR08MB79488ADEC4C4E18C3BA7543DCA599@SJ0PR08MB7948.namprd08.prod.
outlook.com>,
Tom Ekberg writes:
>I have a user that wanted to search issues that have "NO LINE" and 
>autofax. I thought we were using xapien so I entered "no line" AND 
>autofax in the 'all text' part of the search page.

Using native (and possibly others), AND (any capitalization) is a stop word (see backends/indexer_common.py) and it is ignored.  IIRC native is always an AND search, and there is no phrasing (quoted string) support.

So it should return the same as:

  no line autofax

basically it does a (pseudo SQL):

   select messgeid from _words where word == 'no'
   intersect
   select messgeid from _words where word == 'line'
   intersect
   select messgeid from _words where word == autofax

It is a message based intersect not an issue based intersection.

See: https://issues.roundup-tracker.org/issue1155657

>It returned 30 issues where only 1 had both parts of the search and 
>missed another matching issue.

That's unexpected. Is it possible the missed issue has all the words spread across different messages? If so than it's undesirable but expected.

>It turns out that neither xapian nor woosh are installed, so, according 
>to the config.ini the native search is used.
>
>I searched the doc/ directory and the "Roundup User Guide" but didn't 
>find much on the native query language, with regard to 'all text'.

There is no "query language" per se. It's an AND token lookup per unit.  The unit IIRC is message, title or other string field with indexme=yes set.

> [... search resorting to grep of messages elided]

>This query identified the 9 issues that contained "no line". It was a 
>simple process to see which issues in that list also contain autofax 
>using the issue search page.
>
>There must be an easier way.

Whoosh does have a query language.

   https://issues.roundup-tracker.org/issue2551188

but we don't use it. I am not sure what xapian has.

In theory sqlite, postgres and mysql all have full text search support (which can include substring/trigram searching) but there is no code that uses them.

Do you have a suggestion on where to put further documentation on how full text search works in native mode? Which doc would you expect to cover that?

Sorry I don't have better news.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.