Re: Unicode in vCard21

Chris,

You mention that various tools don't create BOM marked files.  What about how
they handle them?  Are they capable of handling your suggestions?

This is a two-fold problem, one the bears fixing on the application side
whenever possible.  As is the custom in formats like XML, be conservative in
what you produce but liberal in what you'll accept.  I'd certainly prefer being
a strict as possible in that it often avoids layers of special-case and error
handling code.

-Bill Kearney

----- Original Message -----
From: "Chris Page" <[email protected]>
To: "IMC vCard" <[email protected]>
Sent: Wednesday, January 15, 2003 11:21 AM
Subject: Re: Unicode in vCard21


>
> On Tuesday, Jan 14, 2003, at 21:16 US/Pacific, Generation NeXT wrote:
>
> > I was reading through the spec, and I was not exactly clear as to how
> > syntactily I could specify UTF-16BE, UTF-16LE, UTF-8, etc.
>
> Right. The spec doesn't really specify anything about file formats,
> which is my big complaint about it. As a result, there are files out
> there that, for example, use a Windows character encoding, but don't
> put any encoding information in the file. So you have to use ad hoc
> methods to determine the encoding of the file contents.
>
> The IETF standard is for embedding vCards in MIME wrappers, where the
> MIME headers are used to specify the encoding. It does not define how
> you might use that spec to store a vCard in a file. You could
> theoretically include the appropriate MIME headers at the start of the
> file, but I suspect you may not find any readers that support that. So,
> again, you're stuck using ad hoc methods to determine whether a file is
> a vCard and what encoding is used.
>
> For UTF, luckily, this can be a little easier. Writers should write a
> BOM, even if using UTF-8 (note that the BOM will be encoded into UTF-8,
> which means it will take up more than two octets). If the file starts
> with a BOM, this should be an unambiguous indication of the UTF
> variant. UTF-8, UTF-16BE, and UTF-16LE will each have a unique series
> of octets for the BOM.
>
> Now, Microsoft's Entourage for Mac writes out UTF-16 files without a
> BOM. The way to detect files like this is to look for the "BEGIN:vCard"
> signature in UTF-16 and see if you get a match (it's just the string
> "BEGIN:vCard" with zeroes in front of each character).
>
> So, I encourage everyone going forward to write Unicode when possible,
> and write a BOM at the start of the file. Only write files using other
> encodings when you need to be compatible with readers that don't handle
> Unicode.
>
> --
> Chris Page - Mac Guy - Palm, Inc.
>