Re: Searching within very large subset of documents

The TermInSetQuery query might be just what I need=E2=80=A6

Essentially, I=E2=80=99m trying to allow a user to provide a file =
listing a subset of documents with an arbitrary document numbers, stored =
as a term in the index. After that, I want them to interactively make =
searches within that subset.

Merci,
-twb

> On Aug 4, 2025, at 11:26=E2=80=AFPM, Adrien Grand <[email protected]> =
wrote:
>=20
> Hi Thomas,
>=20
> Your question suggests that you are creating a huge BooleanQuery to
> identify these documents. A TermInSetQuery should perform better.
>=20
> Doing better would require to better understand what you are trying to
> achieve. For instance if you end up with such a large list of terms =
because
> you're trying to evaluate a join, you may want to look at Lucene's =
support
> for suery-time joins:
> =
https://lucene.apache.org/core/10_1_0/join/org/apache/lucene/search/join/p=
ackage-summary.html#query-time-joins-heading
>=20
> Le mar. 5 ao=C3=BBt 2025, 05:48, Thomas Barr <[email protected]> a =
=C3=A9crit :
>=20
>> I have a medium-sized (~10m) Lucene index and I frequently want to
>> repeatedly search within a subset of around ~100k documents. I can =
increase
>> MaxClauseCount and build up a huge TermQuery, keep that around, then =
build
>> a BooleanQuery out of the result at runtime, but the resulting query =
is
>> quite slow. The now deprecated Filter would have been a good option =
with a
>> BitSet, but that=E2=80=99s deprecated.
>>=20
>> Any thoughts on the best way to do this?
>>=20
>> Thanks!
>> -twb
>>=20
>>=20
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>=20
>> Adrien