Re: Expected developer workflow for maintaining/using custom codecs?

: My experience maintaining custom codecs is that we have to modify them
: whenever we upgrade Lucene, yes. This isn't a problem because we don't have
: dependents consuming our software and building their own from it; these are
: services that are not intended to be further extended.  Or considering
	...
: I guess what I wonder is how frequently is the problem you're describing
: occurring. Is this a theoretical issue, or do you know folks distributing
: custom Lucene codecs whose consumers would update Lucene independently?

It's happening right now, with Solr as the "consumer" of a third-party 
project (That I mentioned before -- see below), attempting to provide a 
custom HNSW vector codec based on an Nvidia CPU acceleration library (not 
just for Solr, but for any other Lucene based applications)

Based on the problems Solr encountered with their custom Codec when trying 
to upgrade from Lucene 10.2->10.3, they've switched their tactic to only 
provide custom KnnVectorsWriter/KnnVectorsReader/KnnVectorsFormat classes 
-- but those classes still (currently) have compile time dependentices on 
org.apache.lucene.codecs.lucene99.* classes -- which could at any time be 
moved to org.apache.lucene.backward_codecs.lucene99.* in a MINOR Lucene 
release.

Hence the question: how is this suppose to work?

How are third-party devs who want to write custom extensions/filters 
around the "default" Codec (or pieces of the default codec) suppose to be 
able to provide any sort of back compat garuntees to Lucene applications 
that want to use their custom codec classes, if Lucene may change the 
package names at any time out from under them?


: > This questions specifically stems from this project...
: >
: >     https://github.com/rapidsai/cuvs-lucene/
: >
: > ...and some issues that popped up here...
: >
: >     https://issues.apache.org/jira/browse/SOLR-17892
: >
: > ...but the confusion is applicable to anyone who might want to write a
: > custom codec and share it with the world.


-Hoss
http://www.lucidworks.com/