Re: Queries on Lucene Replication Approach

Hi, we use Lucene NRT replication in production.

For consistent snapshots, we use SnapshotDeletionPolicy to open a =
snapshot, and then copy the snapshot'ed
files with your tool of choice like rsync. Without a snapshot, I don't =
think such tools work reliably - you can copy
commit metadata without the data it refers to, for example.

We've run Lucene NRT replication in production for about 4 years now.
Overall very happy with it. We intentionally run a simple configuration =
(one indexer, N searchers)
to avoid difficult problems around leader election and network =
partitioning.

Definitely took a bit of learning, but overall we would choose this path =
again.
It is more "hands on" than just spinning up an Elasticsearch cluster - =
but on the other hand
our previous Elasticsearch cluster took a team just just for "care and =
feeding" as the cluster had
split-brain issues etc. Compare to Lucene NRT replication, where a =
single dev and no ops people=20
can easily manage the entire stack, since all state is on a single =
master node.

> On Aug 25, 2025, at 8:36=E2=80=AFAM, Adrien Grand <[email protected]> =
wrote:
>=20
> Hello,
>=20
> If you can guarantee no concurrent updates while you are doing the =
copy,
> then 3rd-party tools that replicate files such as fsync are fine. But
> otherwise or if you have the choice, I would recommend to use Lucene's
> built-in support for replication, which can also work if there are
> concurrent updates to the index.
>=20
> What is NRTLuceneReplication? I cannot find references to it. Lucene's
> replicator module does have NRT (near-realtime) support through:
> =
https://lucene.apache.org/core/10_2_0/replicator/org/apache/lucene/replica=
tor/nrt/package-summary.html
> .
>=20
> On Mon, Aug 25, 2025 at 5:18=E2=80=AFPM sandy A =
<[email protected]> wrote:
>=20
>> Hi Lucene Community Team,
>>=20
>> I have a couple of queries related to Lucene replication and would
>> appreciate your guidance:
>>=20
>> *Query 1:*
>> Is it safe to use tools like *rsync* (on Linux) or *robocopy* (on =
Windows)
>> for copying Lucene segment files from one server to another?
>> I want to understand if there are any potential risks (e.g., partial
>> copies, consistency issues) or best practices recommended for such
>> approaches.
>>=20
>> *Query 2:*
>> I noticed that *NRTLuceneReplication* is still marked as "in =
development."
>> Could you please clarify why it is not yet considered =
production-ready? Are
>> there known limitations or stability concerns that we should be aware =
of
>> before experimenting with it?
>>=20
>> Any insights or suggestions would be really helpful.
>>=20
>>=20
>> With Regards,
>> SanthoshKumar A
>>=20
>=20
>=20
> --=20
> Adrien