choosing between probabilistic and boolean prefixes for terms

Eric Wong <[email protected]>
Newsgroups gmane.comp.search.xapian.general
Message-ID <20180719203223.n3qu5fizixmkgwqk@dcvr>
Hi all,

public-inbox allows searching for git blob names (e.g. "badc0ffee")
in patches.  Initially, I chose to use add_prefix for probabilistic
terms, since I assumed it could be a superset of what boolean
searching offered.  Unfortunately, it doesn't seem to be the case
because stemming is interfering.

So switching to boolean filtering seems to work; and it is
fine for mechanical searches I plan on doing:

  https://public-inbox.org/meta/[email protected]/

Now I wonder, is there a way to get the best-of-both-worlds so
a human can still use wildcards?

public-inbox also allows searches on pathnames, and maybe that
should use boolean filtering, too...

My setup for the query parser isn't anything special:

our $LANG = 'english';
sub stemmer { Search::Xapian::Stem->new($LANG) }

sub qp {
	my ($self) = @_;

	my $qp = $self->{query_parser};
	return $qp if $qp;

	# new parser
	$qp = Search::Xapian::QueryParser->new;
	$qp->set_default_op(OP_AND);
	$qp->set_database($self->{xdb});
	$qp->set_stemmer($self->stemmer);
	$qp->set_stemming_strategy(STEM_SOME);
	$qp->set_max_wildcard_expansion(100);
	$qp->add_valuerangeprocessor(
		Search::Xapian::NumberValueRangeProcessor->new(YYYYMMDD, 'd:'));
	$qp->add_valuerangeprocessor(
		Search::Xapian::NumberValueRangeProcessor->new(DT, 'dt:'));

In any case, all the code is available via:

	git clone https://public-inbox.org/public-inbox
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.