Re: Internationalization

Some thoughts

1) Truncation; you should specify what a receiver SHOULD do when it receives an
incomplete multi-byte sequence; discard the message? use as much as it can?

2) Shortest form; characters can be encoded in more than one way and this
generates a security exposure so shortest fom SHOULD always be generated and
messages not in shortest form SHOULD be discarded.

3) Subset; Unicode is big and mostly unwanted, positively unhelpful even.  Some
RFC have specified a subset but I not seen any one gain acceptance.  I suggest a
recommendation that it be restricted to U+0020..U+007F unless there is a good
reason not to.

4) Reference; if you want a reference for Unicode, you have a choice of the
website or the Addison-Wesley book

5) Which strings can be in UTF-8?  As fewe as possible IMHO, ie only where the
string
needs to make sense in another language, not in names or other short strings
that are used as a handle, a reference, for which the meaning and the language
usually becomes academic.

While the IETF mandates UTF-8, I believe it has not done enough to support those
writing I-Ds;  I did raise this on the main list last December and did not
change my views as a result.  There is expertise but often we are left to
founder; I
think that one of the better bits of floundering recently is in
http://www.ietf.org/internet-drafts/draft-ietf-syslog-protocol-17.txt

Tom Petch

----- Original Message -----
From: "Joel M. Halpern" <[email protected]>
To: <[email protected]>
Sent: Thursday, September 28, 2006 8:41 PM
Subject: Internationalization


> The Forces LFB definition allows LFBs to include strings.  That seems
> pretty useful.
> However, that seems to raise the i18n question.  We probably need to
> say something about this.
> For the properties section, I am declaring that the string length is in
octets.
>
> Currently, the actual definition says that strings are null terminated ASCII.
> That is liable to get us in trouble.
> But allowing multi-byte characters, and other things, is going to get
> us into termination problems.
>
> Now, string are always shipped with lengths.  So we could get rid of
> the null termination.
> And we could define them to be UTF-8, and declare that the length is
> the maximum number of octets, not the maximum number of characters.
>
> I don't know if that is sufficient.  But it would probably help.
>
> Feedback ASAP.  This will affect progression of the document.  There
> are folks out there who watch very carefully for this sort of issue.
>
Yup, it stirred me from my slumbers.
> Yours,
> Joel