Re: UTF-8 full support in LDIF / LDIF v2

On Jun 30, 2009, at 3:13 PM, Michael Ströder wrote:

> Kurt Zeilenga wrote:
>>
>> On Jun 18, 2009, at 3:57 AM, Michael Ströder wrote:
>>
>>> Stepping back a bit from the details of the interesting Unicode  
>>> issues
>>> posted here I wonder what the general strategy of the IETF regarding
>>> these issues is?
>>
>> Punt.
>>
>>> I remember discussions on the ietf-pkix mailing list
>>> mentioning problems like these (e.g. when displaying subject names  
>>> of
>>> X.509 certs) without any real solution.
>>>
>>> I think any system which takes (user) input, decodes it to a Unicode
>>> code point sequence and display it to the user is affected by issues
>>> with BIDI, combining characters and duplicate Unicode points.
>>
>> Yes.  The IETF tends to punt such issues to the user interface
>> development community.  The IETF tends to restrict itself to design  
>> of
>> protocols not design user interface (though the IETF does try to
>> document user interface issues, especially those with security  
>> impact).
>>
>> I think of LDIF as an alternative encoding of protocol data units,  
>> used
>> for out-of-band transfer data between protocol peers.  That is, I  
>> punt
>> the "user" as far as I can.
>
> I'm not sure whether I fully understand the usage of the word "punt".
> (Bear in mind I'm not a native English speaker.)
>
> I agree that LDIF is just an alternative encoding of protocol data
> units. So lifting the ASCII limitation in LDIF would IMO not introduce
> any other problem a LDAP client with user interface does not already
> have today (despite the new-line issue).

Removing the ASCII restriction will mean that users and systems will  
need to be far more careful in how they transfer LDIF data.   Today,  
one can email LDIF files about, FTP them without putting clients in  
"binary" mode, etc, because of the ASCII restriction.  Without this  
restriction, users and systems will have to be quite careful to ensure  
transfers preserve the LDIF data octet-for-octet.

>
>> Others see LDIF as a user display format
>
> Strictly speaking nobody said that. ;-}
>
> Seriously there's another layer between the LDIF data and the user -  
> the
> user interface. So if the IETF does not care about the user  
> interface we
> also don't have to care about that regarding LDIF.

It's not that the IETF doesn't care about UIs, it's just that UIs are  
typically outside the scope of expertise and areas of work the IETF  
normally undertakes.

>
> Whether a particular LDIF is displayable by any means in some  
> localized
> context is not something we have to solve at this LDIF-formatted PDU
> level because we also don't do it for normal LDAP PDUs.

Yes.  But folks here seem to be calling for LDIF to be displayable/ 
editable.

>
>> and user input format for LDAP data.
>
> If the user enters data into an input field most times he uses his
> normal keyboard and the display shows the input using fonts  
> installed on
> his system which matches his language.

It's generally presumed that the user is using a client which is aware  
of the application specific semantics and syntaxes of the data he is  
entering, and if not, the user is obligated to do the work on his own  
to ensure application-specific semantics and syntaxes are met.

> I don't think that there is
> really a new problem introduced except a BIDI problem because the
> attribute type names are written left-to-right. Maybe even this  
> could be
> solved in a decent user interface.

A decent UI would not display attribute type names, but localized  
strings convening the appropriate semantics.  For instance, a decent  
UI ought not display "SN" (or the OID identifying the SN attribute  
type) but the local equivalent to "Surname", and then ensure valued  
entered met the syntactical requirements of SN attribute type.

> So I'd lift the ASCII limitation but I'm not keen on having to deal  
> with
> the "here" document scheme.

I would have less problem replacing the current "safe character"  
restriction with a broader set of "safe characters".  However, I  
rather not spend my time arguing over which particular characters are  
"safe enough" (for interoperability in data exchange).

-- Kurt