Re: Radixor-Lucene: external stemming adapter for Lucene 10

Hi Uwe, hi Mikhail,

To clarify a few practical points: Radixor has no dependencies, and 
Radixor-Lucene depends only on Radixor. On the data side, the runtime 
can work with source dictionaries, with compiled stemming data, or with 
deployment-supplied resources if the data are not packaged directly in 
the module. Radixor-Lucene already exposes those loading modes through 
configuration, so the practical choice is mainly between larger source 
data and smaller precompiled resources with a fixed reduction choice, 
depending on what the community would consider preferable.

One aspect that may be technically interesting in the Lucene context is 
that Radixor is not limited to a single output form. It can emit 
multiple candidate stems for the same token position, and those 
alternatives can also be ranked or weighted.

What I would be interested in understanding better is whether that kind 
of model can be expressed cleanly within Lucene’s existing analysis and 
retrieval model, or whether it would require additional support deeper 
in the indexing model.

Best regards,
Leo



On 4/20/26 3:05 PM, Mikhail Khludnev wrote:
> Hello,
> IIRC both servers (Elasticsearch & Solr) have external analysis chains 
> and also document's field can carry TokenStream from an external 
> analysis chain. So, an integration seems possible without bringing new 
> dependencies.
>
>
> On Mon, Apr 20, 2026 at 12:12 PM Uwe Schindler via dev 
> <[email protected]> wrote:
>
>     Hi,
>
>     in general we are open to include external stemmers into the
>     Lucene code base (maybe only the adaptor), but the external
>     depenencies should be kept minimal (not including a ton of utility
>     libraries like Apache commons-foobar added as transitive
>     dependencies). As far as I understand, the stemmer classes are not
>     "ready-to use", so to actually include the filter into your
>     analysis chain you need to add those data files possible created
>     with the Radixor framework, correct? If we need to shit with many
>     data files for all those languages, we may need to think about a
>     way to regenerate them in our code and ship them to have
>     ready-to-use stemmers for supported languages.
>
>     Small comment about the Lucene code: It has a services file for
>     the factory, but no module-info.java class file yet. It would be
>     great to add a module descriptor - also for the Radixor framework,
>     too, because Lucene is module conformant. This will be mandatory
>     when it needs to be included into Lucene.
>
>     Uwe
>
>     Am 19.04.2026 um 23:30 schrieb Leo Galambos:
>>
>>     Hello Lucene developers,
>>
>>     I would like to share a new external project that may be relevant
>>     to the Lucene analysis community: Radixor-Lucene.
>>
>>     Radixor-Lucene is an adapter for using the Radixor stemmer in
>>     Apache Lucene 10. Radixor builds on ideas from the historical
>>     Egothor stemming line, with a focused implementation aimed at
>>     compact compiled runtime data, deterministic lookup, and
>>     practical extension with domain-specific vocabulary.
>>
>>     It may be relevant in cases where one would otherwise consider
>>     Stempel-style stemming or Porter-family stemmers. It is a
>>     stemming toolkit, not a full morphological analyzer.
>>
>>     Project links:
>>
>>      *
>>
>>         Radixor: https://leogalambos.github.io/Radixor/
>>
>>      *
>>
>>         Radixor-Lucene: https://github.com/leogalambos/Radixor-Lucene
>>
>>     I would be interested in feedback on whether this is best kept
>>     purely as an external adapter, or whether the Lucene community
>>     would see value in it as part of a broader modernization path for
>>     the Stempel-style stemming space.
>>
>>     Best regards,
>>     Leo Galambos
>>
>>
>     -- 
>     Uwe Schindler
>     Achterdiek 19, D-28357 Bremen
>     https://www.thetaphi.de
>     eMail:[email protected]
>
>
>
> -- 
> Sincerely yours
> Mikhail Khludnev