Re: Upgrade Codec 99 to 101 / 102

Hey Michael,

Right now it won't quantize byte vectors that are provided. It just passes
them and treats them like normal.

In the future, we would like to quantize the bytes as well!

On Wed, Sep 10, 2025, 10:56 AM Michael Wechner <[email protected]>
wrote:

> Hi Uwe
>
> Thanks for your feedback!
>
> I am using now:
>
> private final int maxDimensions = 16384;
>
> Codec codec = new Lucene101Codec() {
>     @Override    public KnnVectorsFormat getKnnVectorsFormatForField(String field) {        var delegate = new Lucene102HnswBinaryQuantizedVectorsFormat();
>         return new DelegatingKnnVectorsFormat(delegate, maxDimensions);
>     }
> };
> return codec;
>
> This seems to work fine, but I do not understand whether there is a difference anymore between float32 and int8 vector values,
> or whether Lucene102HnswBinaryQuantizedVectorsFormat always "quantizes" the values?
>
> Thanks
>
> Michael
>
>
> Am 10.09.25 um 10:19 schrieb Uwe Schindler:
>
> Hi,
>
> I think the best is to check the source code of default codec of the
> upgrade version and start from there. I agree, there should possibly a
> documentation available that gives the default componenets used in the
> default codec of a given release.
>
> Uwe
> Am 09.09.2025 um 22:49 schrieb Michael Wechner:
>
> Hi
>
> With Lucene 9.12.0 I set my own custom max vector dimension using
>
> Codec codecInt8 = new Lucene99Codec() {    @Override    public KnnVectorsFormat getKnnVectorsFormatForField(String field) {        var delegate = new Lucene99HnswScalarQuantizedVectorsFormat();        log.info("Vector Value Type: int8, Maximum Vector Dimension: " + maxDimensions);        return new DelegatingKnnVectorsFormat(delegate, maxDimensions);    }};
>
> and
>
> Codec codecFloat32 = new Lucene99Codec() {
>     @Override    public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
>         var delegate = new Lucene99HnswVectorsFormat();
>         log.info("Vector Value Type: float32, Maximum Vector Dimension: " + maxDimensions);
>         return new DelegatingKnnVectorsFormat(delegate, maxDimensions);
>     }
> };
>
> I am a little confused re which Codec / Vector Format classes I should use when upgrading to Lucene version 10.2.2
>
> Any hints would be much appreciated!
>
> Thanks
>
> Michael
>
>
> --
> Uwe SchindlerAchterdiek 19, D-28357 Bremen <https://www.google.com/maps/search/Achterdiek+19,+D-28357+Bremen?entry=gmail&source=g>https://www.thetaphi.de
> eMail: [email protected]
>
>