RE: Operations crossing replication contexts

Hmmm, just when I was about to call to see if all of the comments were
finished on URP, up pops this message. So I have a quick question to John,
Steven, and others that are interested in this thread: is it your aim to
resolve this question as part of the URP Last Call, or is this something
that you think affects other documents than URP?

thanks and kind regards,
John

-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of John McMeeking
Sent: Friday, September 28, 2001 6:52 AM
To: [email protected]
Subject: Operations crossing replication contexts



Thought I'd move a discussion between Steven Legg and myself into the
public eye...

This particular discussion is about how to deal with operations that cross
replication contexts:
- move an entry/subtree from one replication context to another
- delete/rename entries that ancesters of replication contexts (mentioned
at March IETF meeting)

It hasn't progressed to the point of proposing concrete solutions just
yet.

With respect to moving entries between replication contexts, we note that
this has to be initiated on a server that has contains updateable replicas
of both contexts, else the server would fail the client request.  However,
others servers may have replicas of only one of the contexts.  Perhaps
such
an action shouldn't be allowed. Perhaps it should be allowed only if the
initial server can detect (based on replicaSubebtries) that both contexts
are present on all relevant servers.  Or it requires a bit more invention.

With respect to the second point, this has to do with how a server knows
what the boundaries of a replication context are, then how the server(s)
should behave.  An engineering group was formed at the March IETF meeting.
I couldn't attend the first get together, and I am not aware that anything
happened afterwards.  An example of the situation:
- renaming the ancester of a relication context has the effect the
renaming
the child replication context.  The server where the ancester is located
may not be aware that the child context even exists.  Should this be
allowed?  How would replication handle it?


John  McMeeking
IBM

----- Forwarded by John McMeeking/Rochester/IBM on 09/28/2001 08:31 AM
-----

                    "Steven Legg"
                    <steven.legg@adac       To:     John
McMeeking/Rochester/IBM@IBMUS
                    el.com.au>              cc:
                                            Subject:     RE: Single lost &
found (RE: WG Last Call on URP
                    09/28/2001 12:01         I-D)
                    AM
                    Please respond to
                    steven.legg






John,

Apologies for taking my time responding.

> Pondering this a bit more...
>
> The problem of cross replication context operations could be
> rather crudely
> solved by stating that LDUP does not support moving entries
> into or out of
> areas of replication.  You must delete entries from the source, and
> recreate them in the target.  Hopefully an implementation
> would provide
> some tools to make that exercise simpler.

I'm not sure requiring delete/insert for moves between replication
contexts
actually solves more problems than it creates.

Moves by delete/insert are something I want to avoid, in part because
it makes moving subtrees expensive and dangerous. Expensive because
every entry in the subtree has to be deleted and reinserted, and
dangerous because other updates to entries in the subtree could get
lost or corrupted. Recovery from system crashes in the middle of a
subtree move also has to be dealt with.

Since the propagation of updates between servers is not instantaneous,
we would still have to deal with the possibility of a clash between
an already propagating move update and a propagating administrative
action to repartition the replication area. Propagation delays and
conflicting changes also make it possible for a server to perform
what looks to it like an intra-context move but which to another server
appears to be an inter-context move. This can happen even if both
servers have full knowledge of the replication context boundaries.

I also have concerns with any solution where consumer DSAs making
secondary shadowing agreements can hamstring the operations of the
original supplier DSA, i.e. by creating additional replication context
boundaries across which the original supplier can no longer do simple
moves. In X.500 replication (which we implement) a supplier DSA doesn't
need to have any knowledge of the secondary replication agreements
a consumer might make. This is a good thing for scalability in large
distributed environments.

My preferred approach is to have a go at solving the problems with
entries moving in and out scope for partial replicas, and apply that
solution to entries moving in and out of replication contexts, since
the problem spaces appear to be related.

I haven't been devoting much time to thinking about replication in recent
times but an LDUP implementation is on the agenda for our next release
so I will be spending more time on it. In the last couple of days I've
been following up on the idea of tagging entries with the replication
areas in which they appear. This approach is looking promising.
I'll let you and the rest of the working group know what I discover.

>
> That still leaves the question of how a server knows when an operation
> crosses such boundaries.  I asked that question at the March
> IETF meeting
> in Minneapolis.  An engineering group was formed (to include LDAP-EXT
> folks), and far as I can tell, never met.  I've had various
> thoughts on the
> matter.

Except for the effects of propagation delays I don't see a serious problem
in deciding if a move crosses a boundary, but maybe we're not talking
about the same thing. A server can only action a move operation if it
masters both the old and new superior entries. If it masters those
entries then it will have the necessary information about the replication
context of each. A new superior entry in a child replication context
held in the same server presents no problem. If the child replication
context is primarily held in a different server then the superior server
either doesn't master the new superior so it will disallow the move, or
it replicates the subordinate server's replication context in which
case it has the necessary information.

> -  One idea was that each server participated in a
> domain-like entity,where
> the definition of the domain resides in LDAP as a special area of
> replication.  Each server then can has knowledge of all the
> replication
> contexts and all the servers.  Alternately, x.500 glue
> entries might be
> used.  My sense is that this type of approach implies more distributed
> directory function than the LDUP members want.
> - At a minimum, it seems that each server needs to know the
> bounds of the
> replicas it hosts, that is the names of subordinate
> replication contexts.
> I envision this being done through something like an
> operational attribute
> of a replication context.  When a child area of replication
> is created, the
> "childReplicationContext" attribute of the parent replication
> context is
> updated, and that change replicated to other replicas of the parent.
> Deleting a child replication context may require
> administrative action, as
> I don't see how any single server could know that it held the last
> remaining copy of the child area of replication (unless this
> happens on a
> server that holds both the parent and child replication contexts).  I
> suppose creating a child area of replication might require
> administrative
> action too, unless we require that the child be created on a
> server that
> also holds the parent.  And as I think this through more, I
> suppose that
> some sort of glue entry would do the trick here (possibly
> with a reference
> to some server containing that replication context).
>
> Either way, it implies that something holds knowledge of the entire
> replication topology, and that a tool hide as much of the nastiness as
> possible.
>
> Any thoughts?  And I suppose I really ought to put this
> discussion on the
> LDUP mailing list.  Do you agree?

It wouldn't hurt.

Regards,
Steven