Re: Unicode in vCard21

On Tuesday, Jan 14, 2003, at 21:16 US/Pacific, Generation NeXT wrote:

> I was reading through the spec, and I was not exactly clear as to how 
> syntactily I could specify UTF-16BE, UTF-16LE, UTF-8, etc.

Right. The spec doesn't really specify anything about file formats, 
which is my big complaint about it. As a result, there are files out 
there that, for example, use a Windows character encoding, but don't 
put any encoding information in the file. So you have to use ad hoc 
methods to determine the encoding of the file contents.

The IETF standard is for embedding vCards in MIME wrappers, where the 
MIME headers are used to specify the encoding. It does not define how 
you might use that spec to store a vCard in a file. You could 
theoretically include the appropriate MIME headers at the start of the 
file, but I suspect you may not find any readers that support that. So, 
again, you're stuck using ad hoc methods to determine whether a file is 
a vCard and what encoding is used.

For UTF, luckily, this can be a little easier. Writers should write a 
BOM, even if using UTF-8 (note that the BOM will be encoded into UTF-8, 
which means it will take up more than two octets). If the file starts 
with a BOM, this should be an unambiguous indication of the UTF 
variant. UTF-8, UTF-16BE, and UTF-16LE will each have a unique series 
of octets for the BOM.

Now, Microsoft's Entourage for Mac writes out UTF-16 files without a 
BOM. The way to detect files like this is to look for the "BEGIN:vCard" 
signature in UTF-16 and see if you get a match (it's just the string 
"BEGIN:vCard" with zeroes in front of each character).

So, I encourage everyone going forward to write Unicode when possible, 
and write a BOM at the start of the file. Only write files using other 
encodings when you need to be compatible with readers that don't handle 
Unicode.

-- 
Chris Page - Mac Guy - Palm, Inc.