Re: UTF-8 full support in LDIF / LDIF v2

Michael,

Michael Ströder wrote:
> Steven Legg wrote:
>> Kurt Zeilenga wrote:
>>> On Jun 16, 2009, at 4:54 PM, Steven Legg wrote:
>>>> Also, X.500 directories already lose something in the translation
>>>> when outputting as LDIF.  For example, the choice in a
>>>> DirectoryString is lost and if that choice is teletexString then
>>>> transcoding wipes out the exact octet encoding. Most of the
>>>> ad-hoc LDAP string encodings are lossy in some respect. Such
>>>> changes are tolerable because the resulting value in LDIF is the 
>>>> same as far as the matching rules are concerned. Unicode
>>>> normalization of the extended LDIF output is a similar situation.
>>> The loss you are talking about is inherent in LDAP not LDIF.  That is,
>>> LDIF does not lose anything (for the LDAP requests it's design to
>>> represent) in translation to/from LDAP.  I don't it is tolerable for
>>> an LDAP intermediate format to "lose" LDAP information.
>> A "here" document mechanism where the UTF-8 character sequence between
>> the end of the introducer and the beginning of the terminator is the
>> literal directory attribute value without any modification would satisfy
>> the no loss requirement.
> 
> Steven, like Kurt I cannot see how the loss of the choice in a
> DirectoryString stored in a X.500 directory is prevented by the "here"
> document mechanism

It isn't. My comment was in the context of LDAP to LDAP interchange.

 > since it's simply already lost when retrieving the
> DirectoryString attribute via LDAP. (Well, one might think about
> inventing a scheme with attribute sub-types to keep the choice...)

There is a scheme already. I could create LDIF dumps where every attribute
of DirectoryString syntax is output with ";binary", which would necessarily
cause the values to be base64 encoded. However, I don't think there are
many directory implementations that could cope with that so I accept that
X.500 to LDAP to LDIF is essentially lossy (but not in any way that has
proven to be critical). Consequently I'm not too concerned if LDAP to LDIFv2
isn't perfect either, though the "here" mechanism described above does at
least provide for lossless LDAP to LDIFv2 conversion.

The potential for an inadvertent change of normalization in the LDIFv2 if
it is edited doesn't overly concern me. Stringprep takes care of it for
matching purposes and any client that expects attribute values to be in,
or remain in, a particular normalization form is asking for trouble. The
values could be modified by some other client that changes the normalization
during editing and I wouldn't count on every directory implementation
preserving the exact character sequence it is given (though mine does).
If a client needs the values to be in a particular normalization form it
should do the conversion itself.

Regards,
Steven

> 
> Ciao, Michael.