Re: Proposing 10.5.1 bugfix release
Alan Woodward <[email protected]> Wed, 1 Jul 2026 13:05:35 +0100
| Newsgroups | gmane.comp.jakarta.lucene.devel |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_D44028B5-F9CC-45FF-9737-FE06616523DE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Oh that=E2=80=99s a good idea - IIRC the monitor already does that. In = which case we probably don=E2=80=99t need a bugfix release (although I = think going back to lazy allocation is probably the right thing to do in = any case). > On 1 Jul 2026, at 12:49, Adrien Grand <[email protected]> wrote: >=20 > I see, this is a 50kB allocation per segment, which is fine under = normal usage, but becomes noticeable with percolator queries which = create a new MaxScoreBulkScorer for every document? >=20 > In general, bulk scorers will want to allocate large arrays/bit sets = to help with bulk processing of documents, some other bulk scorers do = this as well: BatchScoreBulkScorer, BlockMaxConjunctionBulkScorer, = DenseConjunctionBulkScorer, DisjunctionMaxBulkScorer. I wonder if a = better fix would be to disable bulk scoring for percolator/monitor-style = usage and force doc-at-a-time evaluation by using ScorerSupplier#get() = (possibly wrapped in a DefaultBulkScorer if you'd like to consume hits = via the BulkScorer API while still doing doc-at-a-time evaluation) = instead of ScorerSupplier#bulkScorer(). >=20 > On Wed, Jul 1, 2026 at 12:40=E2=80=AFPM Alan Woodward = <[email protected] <mailto:[email protected]>> wrote: >> Hi all, >>=20 >> We=E2=80=99ve found a regression in 10.5.0 due to eager allocation of = large array buffers in MaxScoreBulkScorer - fix proposed here: = https://github.com/apache/lucene/pull/16316 >>=20 >> This particularly hits boolean queries with an expensive two-phase = subclause (in our case, some percolator queries got a lot slower). I = think it probably warrants a 10.5.1 bugfix. >>=20 >> - Alan >=20 >=20 >=20 > -- > Adrien --Apple-Mail=_D44028B5-F9CC-45FF-9737-FE06616523DE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html aria-label=3D"message body"><head><meta http-equiv=3D"content-type" = content=3D"text/html; charset=3Dutf-8"></head><body = style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;">Oh that=E2=80=99s a good idea - IIRC the = monitor already does that. In which case we probably don=E2=80=99t = need a bugfix release (although I think going back to lazy allocation is = probably the right thing to do in any case).<br = id=3D"lineBreakAtBeginningOfMessage"><div><br><blockquote = type=3D"cite"><div>On 1 Jul 2026, at 12:49, Adrien Grand = <[email protected]> wrote:</div><br = class=3D"Apple-interchange-newline"><div><div dir=3D"ltr">I see, this is = a 50kB allocation per segment, which is fine under normal usage, but = becomes noticeable with percolator queries which create a new = MaxScoreBulkScorer for every document?<div><br></div><div>In general, = bulk scorers will want to allocate large arrays/bit sets to help with = bulk processing of documents, some other bulk scorers do this as = well: BatchScoreBulkScorer, BlockMaxConjunctionBulkScorer, = DenseConjunctionBulkScorer, DisjunctionMaxBulkScorer. I wonder if a = better fix would be to disable bulk scoring for percolator/monitor-style = usage and force doc-at-a-time evaluation by using ScorerSupplier#get() = (possibly wrapped in a DefaultBulkScorer if you'd like to consume hits = via the BulkScorer API while still doing doc-at-a-time evaluation) = instead of ScorerSupplier#bulkScorer().</div></div><br><div = class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" = class=3D"gmail_attr">On Wed, Jul 1, 2026 at 12:40=E2=80=AFPM Alan = Woodward <<a = href=3D"mailto:[email protected]">[email protected]</a>> = wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px = 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div>Hi = all,<div><br></div><div>We=E2=80=99ve found a regression in 10.5.0 due = to eager allocation of large array buffers in MaxScoreBulkScorer - fix = proposed here: <a = href=3D"https://github.com/apache/lucene/pull/16316" = target=3D"_blank">https://github.com/apache/lucene/pull/16316</a></div><di= v><br></div><div>This particularly hits boolean queries with an = expensive two-phase subclause (in our case, some percolator queries got = a lot slower). I think it probably warrants a 10.5.1 = bugfix.</div><div><br></div><div>- = Alan</div></div></blockquote></div><div><br = clear=3D"all"></div><div><br></div><span = class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"ltr" = class=3D"gmail_signature">Adrien</div> </div></blockquote></div><br></body></html>= --Apple-Mail=_D44028B5-F9CC-45FF-9737-FE06616523DE--