Re: libvc 003 released with vCard 2.1 compatibility and question for the public

Quoteing [email protected], on Fri, Jun 20, 2003 at 09:48:02PM -0700:
> Wow.  I am impressed.  I didn't know there was ruby
> support out there.

Well, see how well it works for you before being too impressed!

> But don't you have to check the BOM to determine if
> the double-byte is in little endian or big endian?

I don't know anything about BOMs, other than they exist. Have you
seen a vCard with one? Apple's don't have them, they look like:

0000000: 0042 0045 0047 0049 004e 003a 0056 0043  .B.E.G.I.N.:.V.C
0000010: 0041 0052 0044 000d 000a 0056 0045 0052  .A.R.D.....V.E.R
0000020: 0053 0049 004f 004e 003a 0033 002e 0030  .S.I.O.N.:.3...0
0000030: 000d 000a 004e 003a 004c 0065 0062 0065  .....N.:.L.e.b.e

I'm just checking if the first char is 0, and if so, since it is
definitely not ascii, or latin-*, assuming it's ucs-2.

I just tried this:

irb(main):005:0> s = "\0b\0e"
"\000b\000e"
irb(main):006:0> Iconv.iconv('ascii', 'ucs-2', s)
["be"]
irb(main):007:0> s="b\0e\0"
"b\000e\000"
irb(main):008:0> Iconv.iconv('ascii', 'ucs-2', s)
Iconv::IllegalSequence: "b\000e\000"
  from (irb):8:in `iconv'
  from (irb):8


So, iconv(), at least, doesn't like the opposite byte order.

> I know Apple doesn't support the BOM, but they seem to somehow handle
> both cases.  I need to experiement though to find out exactly.

Could you send me some samples of the other byte order, or ones with
BOMs? I'll try and decode them.

> If you can do that in an hour, that's impressive.  I

I'm not doing the unicode converting myself, I'm using iconv from
ruby1.8, you can get it for 1.6 from ruby-sumo, or shim-ruby.

It should correctly deal with BOMs, and I think I've seen it doing so.

> was thinking of playing with IBM's open source
> freeware library for Unicode, as this would be a more
> cross-platform solution.

I don't know anything about IBM freeware, but many unix systems have the
iconv() library function, if your system doesn't have it try;

  http://www.gnu.org/software/libiconv/

Ruby's Iconv uses iconv(), of course!

> Though this is down the road.  I think all too often
> vendors ignore multi-lingual and murder even UTF-8
> streams. :-/  So, hopefully some open source solutions
> can help in that regard.

I think vCard 3.0 murdered multi-lingual support by assuming that vCards
would only be transported in MIME wrappers, so they took characterset
information out of it, so now you have no way to know the characterset
of a vCard sitting in a stand-alone file.

Cheers,
Sam