Re: search engines

Peter Münster <[email protected]> writes:

Hi Peter,

I'm filling in the mu/Xapian row.

> Perhaps we could make a survey and a feature table?
>
> I try to start such a table:
>
> engine   substrings       regex   UTF-8   encrypted messages
> --------------------------------------------------------------
> notmuch  start of word    no      yes     yes
> mairix   yes              no      no      no

mu/Xapian  start of word    kindof  kindof  don't know

The "kindof" at regex means: you can specify regex searches with mu by
enclosing you query with /../, e.g., /.*foo.*/ but actually Xapian
doesn't support regexes so mu does some trickery here to make it
"kindof" work.  Basically (as I understand it), it asks Xapian to give
the results for the fixed-string parts (like foo) and then filters those
results with the regexps.

So basically, when you search for subject:/^Emacs.*/ it will actually
retrieve all mails with the subject containing Emacs no matter where,
i.e., the ^ anchor does not work.  That's because the Xapian DB delivers
results for both phrases (the complete subject) as well as words (each
word of the subject separately) where the latter has a match for ^Emacs.

Also spaces don't work in regex searches, i.e., subject:/foo bar/ won't
find anything, so you have to use /foo.bar/ where you'll also find
"foo-bar" etc.

Despite those things, you can use regexes to circumvent the "start of
word" substring matching, e.g., subject:/abled/ will find all mails with
a subject containing "enabled", "disabled", etc.

The "kindof" with UTF-8 means that I think it works in general (I think
notmuch uses Xapian, too) but there seems to be some trickery in here,
too.  When I search for from:/[ÄÖÜäöü]/ I get mails from people with
AOUaeu in their name, too (and not neccessarily ae, oe, ue), so it seems
that Umlauts and their "base character" are treated the same.  When
searching for from:/ß/ I only get mails from people with a real ß in
their name...

Bye,
Tassilo