Re: Content Synchronization Operation + Transactions

On 8.7.2015 17:32, Petr Spacek wrote:
> On 29.6.2015 14:23, Howard Chu wrote:
>> Petr Spacek wrote:
>>> On 3.6.2015 09:35, Petr Spacek wrote:
>>>> Hello,
>>>>
>>>> it seems to me that LDAP Transactions (RFC 5805) are not perfectly integrated
>>>> into Content Synchronization Operation (RFC 4533).
>>>>
>>>> Mainly the client has no idea what changes were part of single transaction. As
>>>> a result, client cannot replicate transactions reliably, especially when
>>>> connection drops in the middle of persist phase of Content Synchronization
>>>> Operation.
>>>>
>>>> Also some clients could use the information where LDAP transaction
>>>> started/ended for application-level transactions (if the application uses the
>>>> data in real-time) or possibly some optimizations (if some application-level
>>>> work needs to be done after each group of updates).
>>>>
>>>> I'm going to sketch -00 draft which will attempt to address this, probably by
>>>> adding new messages to indicate where transaction started and ended.
>>>> Naturally, this makes sense only for persist phase.
>>>
>>> The 00 version of the draft is now available. I would really appreciate any
>>> feedback!
>>>
>>> Particularly, I'm not very sure that "transaction identifier" in "End
>>> Transaction Notification Message" is the best idea.
>>
>> Assuming a unique transaction identifier per transaction, I see no reason you
>> couldn't allow interleaved transactions during persist stage. (Well, you would
>> need to send the txnID along with every message, so that may be excessive
>> overhead.) In OpenLDAP transactions would not be interleaved at execution time
>> anyway so perhaps it's a non-issue.
> 
> Main reason for using extra messages as transaction boundaries was ease of
> implementation on the client side. It can be quite awful to deal with multiple
> concurrent transactions on the application level. The assumption here is that
> transaction commit should be quite quick so serialization should not add a big
> performance penalty (SyncRepl serializes all changes anyway).

After LDAPCon I decided to wipe out transaction identifiers from the draft, so
transaction boundaries are marked by empty Start & End Transaction
Notification Messages.

This would be enough for making SyncRepl useful with transactions in
applications I care about. I will think about writing separate draft for
'ignore own writes', but today I think that it belongs to a separate document.


> Main motivation why tnxID was exposed to SyncRepl users was ability to detect
> own writes. Maybe it is better to hide tnxID completely, use empty Start/End
> Transation Notification messages, and define another control for 'ignore own
> writes when doing SyncRepl'.
> 
> What do you think? I'm not familiar with OpenLDAP internals, so I do not know
> how hard it is to implement 'ignore own writes' on server side. I assume that
> 'own write' = 'write done by the same BIND DN' but this can be tricky when it
> comes to proxying ...
> 
>> On a slightly related note, I intend to submit an extension to RFC5805 adding
>> a Prepare Transaction request. This would enable 2 phase commit, allowing us
>> to support transactions spanning multiple DSAs. With OpenLDAP's back-ldap,
>> back-meta, back-relay, and glue, the capability is sorely needed.
> 
> Good idea!
> 
> Would it be appropriate & possible to clarify what level of 'isolation' is
> assumed in RFC5805, possibly in the same draft? I guess that the
> under-specification will be more visible with 2 phase commit.
> 
> I think that this property is very important and its under-specification can
> cause incompatibilities if other servers decide to implement different levels
> of isolation.

Howard, are you going to clarify isolation level in the new drafts related to
transactions? I think that it would be really useful.

Thank you for your time!

Petr Spacek



-------- Forwarded Message --------
Subject: New Version Notification for
draft-spacek-ldapext-syncrepl-transaction-01.txt
Date: Wed, 18 Nov 2015 09:02:37 -0800
From: [email protected]
To: Petr Spacek <[email protected]>


A new version of I-D, draft-spacek-ldapext-syncrepl-transaction-01.txt
has been successfully submitted by Petr Spacek and posted to the
IETF repository.

Name:		draft-spacek-ldapext-syncrepl-transaction
Revision:	01
Title:		The Lightweight Directory Access Protocol (LDAP) Content
Synchronization Operation with Transactions
Document date:	2015-11-18
Group:		Individual Submission
Pages:		5
URL:
https://www.ietf.org/internet-drafts/draft-spacek-ldapext-syncrepl-transaction-01.txt
Status:
https://datatracker.ietf.org/doc/draft-spacek-ldapext-syncrepl-transaction/
Htmlized:
https://tools.ietf.org/html/draft-spacek-ldapext-syncrepl-transaction-01
Diff:
https://www.ietf.org/rfcdiff?url2=draft-spacek-ldapext-syncrepl-transaction-01

Abstract:
   This document specifies LDAP Control which extends the persist stage
   of the Content Synchronization Operation with information about LDAP
   transaction boundaries.  This information can be used to support
   application-level transactions or for application-level
   optimizations.