Re: FreeTDS and SQL Server 2008 UTF-16 characters

2010/9/24 James K. Lowden <[email protected]>

> Hi Burkhard,
>
> > From all that I learned about this, it seems it would be safe
> > intepreting the UCS-2 characters as UTF-16
> ...
> > I would appreciate your help with such a change... for sure I can help
> > with testing it.
>
> I have a suggestion for a hack.  If you're right and I'm right, it might
> work.
>
> In src/tds/iconv.c, the string constant "UCS-2LE" appears 4 times.  Change
> them to "UTF-16LE".  Recompile.  Make sure you're using GNU libiconv.  Run
> your test.  Watch for smoke.
>
> If UTF-16 really is a superset of UCS-2, it should Just Work.  That change
> doesn't fake anything; it just treats as UTF-16 what would otherwise be
> regarded as UCS-2.  If it works, please update the comments and the
> variable names (and for extra credit, the documentation), post your patch
> and call it a day.
>
> HTH.
>
>
The big problem about moving from ucs2 to utf16 is portability. There are
some implementations (like HP-UX one) which sticks to ucs2 internal format
and BMP 0. In these implementations conversions to/from ucs2 are available
without problems while conversions to/from utf16 are not. We don't handle
utf16 in our replacements library but is not that hard to add it (I already
wrote the patch).

freddy77