Applicability of LDAP Content Sync to loosely coupled replica sets

"Jeff Parham" <[email protected]> Thu, 5 Jun 2003 10:45:15 -0700
Newsgroups gmane.ietf.ldup
Message-ID <6BA5A5D4255F564BBF4BF9E413BF53B80169CFFE@WIN-MSG-05.wingroup.windeploy.ntdev.microsoft.com>
As a server implementer, LDAP Content Sync has some very desirable
properties.  I lean heavily towards state-based designs myself, and the
idea that one could design client sync based purely on server state and
a small amount of client-persisted cookie information is attractive.
However, I have a question about how the general model applies to highly
available, loosely coupled replica sets:

The requirement for high availability of writes encourages adoption of
multi-master replication between DSAs, and geographic data distribution
and other factors tend to encourage models in which these DSAs are
loosely coupled - e.g., an update can be mastered on one DSA, even if
other DSAs in the replica set can't be simultaneously contacted.

This environment leads to "distribution of truth," where no single DSA
can assert the complete, current state of the system - i.e., no single
DSA necessarily holds the results of all updates previously committed to
all DSAs.  The multi-master replication in effect between DSAs in the
replica set continually brings each DSA more up-to-date with respect to
updates made on other replicas, but barring a cessation of updates, any
given DSA can't be asserted to be completely up-to-date.

High availability of DSAs from which clients can synchronize is very
desirable in these environments: if the DSA that a client last
synchronized from is unavailable, the client would like to synchronize
from an alternate DSA.  In doing so, the client very much wishes to
avoid a full reload - to minimize client resource consumption, to
maximize DSA scalability, and to minimize the latency of obtaining the
incremental updates the client desires.

Is LDAP Content Sync intended to be applicable in this case?

Here it does not appear possible for any given DSA to guarantee that its
notion of the "complete" set of objects matching some given criteria is
identical to the set asserted by another DSA.  Given LDAP Content Sync's
method for declaring the set of objects currently matching the client's
scope (by explicitly enumerating their entryUUIDs), might that then mean
that a new object successfully synchronized from DSA1 could be removed
in a subsequent incremental sync from DSA2, where DSA2 has not yet
replicated the new object from DSA1, because DSA2 does not return the
entryUUID for the new object?  If so, how is the client ensured to
eventually converge to a state in which it holds the new object?  (E.g.,
what if the next sync sources from DSA1?)  Does convergence require a
(DSA-indicated) reload?  How would DSA1 otherwise realize and repair the
loss of client state caused by the client's communication with DSA2?

LCUP appears to be immune to this phenomenon.  Objects can be removed
from the client only upon an LCUP full reload, which we believe will be
exceedingly rare.

Thanks,
-J