Re: Proposing 10.5.1 bugfix release

--000000000000a4c5b106558b4908
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I see, this is a 50kB allocation per segment, which is fine under normal
usage, but becomes noticeable with percolator queries which create a new
MaxScoreBulkScorer for every document?

In general, bulk scorers will want to allocate large arrays/bit sets to
help with bulk processing of documents, some other bulk scorers do this as
well: BatchScoreBulkScorer, BlockMaxConjunctionBulkScorer,
DenseConjunctionBulkScorer, DisjunctionMaxBulkScorer.
I wonder if a better fix would be to disable bulk scoring for
percolator/monitor-style usage and force doc-at-a-time evaluation by using
ScorerSupplier#get() (possibly wrapped in a DefaultBulkScorer if you'd like
to consume hits via the BulkScorer API while still doing doc-at-a-time
evaluation) instead of ScorerSupplier#bulkScorer().

On Wed, Jul 1, 2026 at 12:40=E2=80=AFPM Alan Woodward <[email protected]=
> wrote:

> Hi all,
>
> We=E2=80=99ve found a regression in 10.5.0 due to eager allocation of lar=
ge array
> buffers in MaxScoreBulkScorer - fix proposed here:
> https://github.com/apache/lucene/pull/16316
>
> This particularly hits boolean queries with an expensive two-phase
> subclause (in our case, some percolator queries got a lot slower).  I thi=
nk
> it probably warrants a 10.5.1 bugfix.
>
> - Alan
>


--=20
Adrien

--000000000000a4c5b106558b4908
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I see, this is a 50kB allocation per segment, which is fin=
e under normal usage, but becomes noticeable with percolator queries which =
create a new MaxScoreBulkScorer for every document?<div><br></div><div>In g=
eneral, bulk scorers will want to allocate large arrays/bit sets to help wi=
th bulk processing of documents, some other bulk scorers do this as well:=
=C2=A0BatchScoreBulkScorer,=C2=A0BlockMaxConjunctionBulkScorer,=C2=A0DenseC=
onjunctionBulkScorer,=C2=A0DisjunctionMaxBulkScorer. I wonder if a better f=
ix would be to disable bulk scoring for percolator/monitor-style usage and =
force doc-at-a-time evaluation by using ScorerSupplier#get() (possibly wrap=
ped in a DefaultBulkScorer if you&#39;d like to consume hits via the BulkSc=
orer API while still doing doc-at-a-time evaluation) instead of ScorerSuppl=
ier#bulkScorer().</div></div><br><div class=3D"gmail_quote gmail_quote_cont=
ainer"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jul 1, 2026 at 12:40=
=E2=80=AFPM Alan Woodward &lt;<a href=3D"mailto:[email protected]">romse=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pad=
ding-left:1ex"><div>Hi all,<div><br></div><div>We=E2=80=99ve found a regres=
sion in 10.5.0 due to eager allocation of large array buffers in MaxScoreBu=
lkScorer - fix proposed here:=C2=A0<a href=3D"https://github.com/apache/luc=
ene/pull/16316" target=3D"_blank">https://github.com/apache/lucene/pull/163=
16</a></div><div><br></div><div>This particularly hits boolean queries with=
 an expensive two-phase subclause (in our case, some percolator queries got=
 a lot slower).=C2=A0 I think it probably warrants a 10.5.1 bugfix.</div><d=
iv><br></div><div>- Alan</div></div></blockquote></div><div><br clear=3D"al=
l"></div><div><br></div><span class=3D"gmail_signature_prefix">-- </span><b=
r><div dir=3D"ltr" class=3D"gmail_signature">Adrien</div>

--000000000000a4c5b106558b4908--