RE: Supporting Partial Replication

Ed,

Ed Reed wrote:
> John - I think you're onto the right track in determining 
> where the disconnect
> exists.  
> 
> To begin, note that the model only talks about fractional, 
> and not sparse,
> replicas.  
> 
> Fractional replicas are those that hold entries having a subset
> of all the attributes defined on compete replicas holding 
> those entries.  All
> entries of a replication context are present in a fractional 
> replica, but not
> all of their attributes are there.  It is the entries which 
> are fractional.
> 
> Sparse replicas, we agreed, are those where not every entry of the
> replication context are held on the sparse replica.  But every entry
> that IS held is complete.  It is the namespace that is sparse.
> 
> A Sparse and/or Fractional replica (an incomplete replica, in 
> earlier drafts
> of the model) hold some of the entries of the replication 
> context, and they
> may or may not be complete (ie, some entries may be 
> fractional entries).
> In other words, the namespace is sparse and the entries held may be
> fractional.
> 
> We dropped sparse replicas from the Arch Model when we couldn't
> persuade ourselves that we knew how to deal with update vectors and
> purge vectors for sparse replicas.  

That's one of the questions I'm now addressing. It turns out that
fractional replicas (already in the model) and sparse replicas (not
yet in the model) have the same issues with respect to update vectors,
and the same solution.

> 
> Now, I've never contemplated a scenario in which a server holds two
> replicas of same replication context.  There may be utility in
> such a scenario, but it has never occured to me.  Thus, there is one
> update vector for the replication context, on the replicaSubentry.
> 
> The LDUP model in my mind partitions naming contexts with replication
> contexts.  Note that both naming contexts and replication contexts are
> subjects of the DIT, not of any particular DIB.  A replica is a set of
> entries from a naming context held in a particular DIB by a particular
> LDAP server.  There may be multiple replicas of any 
> replication context.
> 
> But as I said, it had never occured to me to consider a DSA holding
> multiple replicas of the same replication context in its DIB.
> 
> Rather, I assumed that a DSA would be considered to hold in its DIB
> all the entries from a replication context that it needed for its own
> purposes, and for the purposes of forwarding updates to other servers
> that it supplies updates to, if any.

I trust that you understand that in advocating an update vector and
replica ID for each area of replication (by which I mean a full or
sparse and/or fractional portion of a replication context) I'm not
suggesting that the data in overlapping areas of replication should
be duplicated. An entry and its attributes are represented only once
in a server even though that entry, or parts of it, may fall within
the scope of more than one area of replication.

> 
> So, subordinate to the replicaSubentry representing the replica in the
> DIB of a DSA, there could be several replicationAgreements - some
> of which place no additional filters on the updates being sent, and 
> some that DO further restrict which attributes (for 
> fractional entries)
> are to be forwarded to the specific other DSA replica pointed to by
> each replicationAgreementSubentry.
> 
> Thus - a replicaSubentry documents the entries held by the DSA and
> whether there are any limits on the attributes held for those entries,
> and replicationAgreementSubentries documents the flow of
> entries (and their attributes, possibly filtered) between a replica
> and another.

There is a degree of redundancy, and therefore scope for consistency
errors, in describing a fractional replica. The same attribute filters
are specified in the replicaSubentry for the fractional replica and in
the replicaAgreementSubentry for propagating updates from the
full replica to the fractional replica. If updateable fractional replicas
are to be allowed then there will also be a replicaAgreementSubentry
for propagating changes from the fractional replica to the full replica,
with the same attribute filters. There would be similar redundancy
for sparse replica specifications.

This redundancy would be removed if areas of replication were explicitly
represented by subentries in a replication context. A replicaSubentry
subordinate to an area of replication subentry indicates that the server
holds the information in that area of replication, and
replicaAgreementSubentries subordinate to the replicaSubentry are assumed
to propagate only changes to the same information. The area of information
subentry is the only place where sparse-ness and fractional-ness needs
to be specified.

> 
> If the set of entries in replication area A held by replica 1 is 
> designated A1 on Server S1, and there is a subset of A held on another
> server S2 designated A2, A1 is a complete replica and A2 is an
> incomplete replica.  It would be redundant to say that A2 and A1
> are both held by S1, though they are, because all the entries of A2
> (being a subset of A) are in A1 (the full set of A).
> 
> Now, A1 on S1 might very well have a replicationAgreement with A2
> on S2 such that A1 only sends what A2 needs because of the filters
> on the replication agreement, or even because of filters on the
> replicaSubentry for A2.
> 
> But the notion that both A1 and A2 would be considered to both
> be on a single server is simply not something I have ever considered.
> I don't know if the model could work in such a scenario or not.
> 
> Ed

Regards,
Steven