Codec-specific index structures on LeafReader
Alan Woodward <[email protected]> Fri, 3 Jul 2026 14:34:35 +0100
| Newsgroups | gmane.comp.jakarta.lucene.devel |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_C606CC0F-D32B-4C38-9086-E52FA9EB127B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
Hi all,
I=E2=80=99ve been catching up with the talks from Buzzwords this year =
and watching this (excellent!) talk from Shailesh =
(https://www.youtube.com/watch?v=3DziRcxCJPU6s) on star-tree indexes in =
OpenSearch made me think again about the way we make index structures =
available on LeafReader. The star-tree index is a novel structure that =
I don=E2=80=99t think it would make sense to add to the base API, but on =
the other hand there isn=E2=80=99t currently an obvious extension point =
that makes it easy to use without doing some hairy casting of LeafReader =
types. We have a similar issue in elasticsearch with bloom filters - =
useful in a restricted set of circumstances, probably not helpful to =
have cluttering up the API, so we end up implementing it in a custom =
Codec and then casting at runtime to see if things are available (with =
the usual problems about FilterLeafReader wrapping that comes with =
relying on casts). This also applies to Codec-specific extensions to =
things like DocValueSkippers.
What do people think about adding a new method to the LeafReader API =
that looks like this:
/** Return a data structure accessed through type T if available =
*/
T getDataStructure(Class<T> structureType) {
return null;
}
Expert users implementing their own Codecs can use this to return =
specialised data structures without polluting the top-level API or =
having to use fragile casts.
Any opinions?=
--Apple-Mail=_C606CC0F-D32B-4C38-9086-E52FA9EB127B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html aria-label=3D"message body"><head><meta http-equiv=3D"content-type" =
content=3D"text/html; charset=3Dutf-8"></head><body =
style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; =
line-break: after-white-space;">Hi all,<div><br></div><div>I=E2=80=99ve =
been catching up with the talks from Buzzwords this year and watching =
this (excellent!) talk from Shailesh (<a =
href=3D"https://www.youtube.com/watch?v=3DziRcxCJPU6s">https://www.youtube=
.com/watch?v=3DziRcxCJPU6s</a>) on star-tree indexes in OpenSearch made =
me think again about the way we make index structures available on =
LeafReader. The star-tree index is a novel structure that I =
don=E2=80=99t think it would make sense to add to the base API, but on =
the other hand there isn=E2=80=99t currently an obvious extension point =
that makes it easy to use without doing some hairy casting of LeafReader =
types. We have a similar issue in elasticsearch with bloom filters =
- useful in a restricted set of circumstances, probably not helpful to =
have cluttering up the API, so we end up implementing it in a custom =
Codec and then casting at runtime to see if things are available (with =
the usual problems about FilterLeafReader wrapping that comes with =
relying on casts). This also applies to Codec-specific extensions =
to things like DocValueSkippers.</div><div><br></div><div>What do people =
think about adding a new method to the LeafReader API that looks like =
this:</div><div><br></div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span>/** Return a data structure =
accessed through type T if available */</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre"> </span>T =
getDataStructure(Class<T> structureType) {</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre"> =
</span>return null;</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span>}</div><div><br></div><div>Expert =
users implementing their own Codecs can use this to return specialised =
data structures without polluting the top-level API or having to use =
fragile casts.</div><div><br></div><div>Any =
opinions?</div></body></html>=
--Apple-Mail=_C606CC0F-D32B-4C38-9086-E52FA9EB127B--