RE: LCUP and PSearch on Sync-And-Persist

Jeff and Rich,

Thanks for your consideration on this topic.

I went back and reviewed the draft to brush up on the general aspects of
the protocol definition.

I still have some questions regarding how the original intentions can be
satisfied.

I see two issues:

1) How are "intermediate updates" to be handled by implementations?  I
suspect that this may be wound up in a server's implementation of the
"state" represented by the "cookie" value and the draft may leave this as
an "excercise for the implementer". If so, I think it would be to
everyone's advantage that the draft indicate "beware - invention required
here" on this topic.  For example: if the "cookie" value changes relatively
often (so that client's presumably don't have to re-start from scratch all
the time), and an update to an entry is received on one thread while a LCUP
session is "in process" on another, it is problematic for the server
implementation to merely send the "updated entry" in the middle of the
"synchronize" data stream for the client.  If this occurs, and the "cookie"
is updated ... depending on the "cookie format", lots of data could be
missed on a re-driven synchronization request.

2) How is a client supposed to know when they're "close"?  I believe the
algorithm you've listed below, using the final "cookie" from the
"synchronizeOnly" session as the "starting point" for the
"synchronizeAndPersist" session, can work (assuming the server understands
this, of course, and the client is careful to re-use the last "cookie"
value from the first LCUP session).  Again, the dangers in "intermediate
updates" should be considered - for example, should server implementations
send "intermediate updates" if "synchronizeOnly" is specified?

Since "cookies" are to be used "across" LCUP sessions, it seems to strongly
suggest that they be related to some form of timestamp.  (I know it's not
required, but establishing "what has changed since I sent this client this
'cookie'" seems to lead one to see that there has to be some time component
to the "cookie" - if only there is a table in the server implementation
which relates the cookie value to some time and space point.  But this gets
into trouble depending on which cookie values are "remembered" by the
client especially if "intermediate updates" flow in the middle of the LCUP
session.

One more point - since the protocol indicates that the response to the
client is in the form of a searchResultEntry, this implies that for entries
with attributes that have a large number of values, small changes to those
attributes will require large data transfers over LCUP sessions.  It would
be good to note this in the LCUP draft as a consideration for
implementors/users of the protocol.

Regards,
Tim Hahn

Internet: [email protected]
Internal: Timothy Hahn/Endicott/IBM@IBMUS or IBMUSM00(HAHNT)
phone: 607.752.6388     tie-line: 8/852.6388
fax: 607.752.3681



                                                                                                                                           
                      "Jeff Parham"                                                                                                        
                      <[email protected]        To:       "Kurt D. Zeilenga" <[email protected]>, "Rich Megginson" <[email protected]> 
                      rosoft.com>                  cc:       "Mark C Smith" <[email protected]>, Timothy Hahn/Endicott/IBM@IBMUS,           
                                                    <[email protected]>                                                                    
                      04/05/2002 11:39 AM          Subject:  RE: LCUP and PSearch on Sync-And-Persist                                      
                                                                                                                                           
                                                                                                                                           
                                                                                                                                           



What is the scenario in which a client needs to do something special on
receipt of the "in sync" notification?

The only one I can see is on an "initial sync," where some clients might
want to know they have a reasonably complete set of search results as
they existed in the recent past (excepting loose consistency due to
replication latency between DSAs, etc.).  In such a case the client
could initially use a synchronizeOnly LCUP exchange with the DSA, then
switch to synchronizeAndPersist for future LCUP interactions.

The example scenario cited by Mircea -- ensuring the client has a
consistent view across many objects -- is not one that can or should be
solved through LCUP.  The solution to this problem is the use of
application versioning of objects.  Many strategies for this exist --
e.g., those detailed at
http://msdn.microsoft.com/library/en-us/netdir/ad/detecting_and_avoiding
_replication_latency.asp.

In short, I am not yet convinced that the currently specified behavior
of LCUP in this area is deficient.

-J

-----Original Message-----
From: Kurt D. Zeilenga [mailto:[email protected]]
Sent: Thursday, April 04, 2002 10:12 AM
To: Rich Megginson
Cc: Mark C Smith; Timothy Hahn; [email protected]
Subject: Re: LCUP and PSearch on Sync-And-Persist


At 09:07 AM 2002-04-04, Rich Megginson wrote:
>"Kurt D. Zeilenga" wrote:
>
>> There are many clients which need to know they are 'in sync'.
>> I note that this is not just after the initial 'sync up' phase,
>> but subsequently.  Also, there are cases where the server
>> needs to update the cookie without sending an update (in the
>> 'persist' phase.
>>
>> I suggest that we add a 'in sync' notification to LCUP.  Basically,
>> just a (intermediate) response PDU which says "you're in sync"
>> and, OPTIONALLY, provides a new cookie.
>>
>> The server should send one when it has sent all changes necessary
>> to 'sync' the client.  The server may send them subsequently to
>> again notify the client it is 'in sync'.
>
>But what exactly does 'sync' mean?

It means that the server has sent all the messages necessary for
the client to have a complete and accurate copy of the requested
(and provided) information.

>Does this mean 'all changes between the time I last connected and the
time I connected for this
>session'?

It means all the changes necessary to have a complete and accurate
copy of the requested (and provided) information.

>Because changes may come in to the server during the sync phase.

The server may send them or queue them or possible use other
approaches.

>What should happen to these changes?  Or should those
>entries be included?  What if the server is under a heavy update load
and never gets to the end of the sync phase?

If it never reaches the 'in sync' condition, it never should sent
the 'in sync' message.

I think we should get away from this 'sync'/'persist' phase
business and look at this simply as a sequence up messages which
'updates' entries of interest.  When the sequence of updates
sent is known to reflect the server's current state (for all
entries of interest), the server sends an 'in sync' message
notifying the client of this.

The initial 'in sync' message should be sent as soon as the
all the changes necessary to be 'in sync' have be sent.
Subsequent 'in sync' messages can be sent whenever the
server desires to send them.

Kurt
Kurt