Re: FreeTDS and SQL Server 2008 UTF-16 characters

[email protected] wrote:

[very nice problem statement ommitted]

> FreeTDS doesn't seem to support such characters that are beyond the BMP.
> In the FreeTDS dump file, I can see the UTF-16 byte sequence coming in
> correctly from the SQL Server 2008, but then apparently the characters
> are converted from UCS-2LE to my client character set (UTF-8). I would
> expect it to convert from UTF-16 to UTF-8.

The library is designed to handle server<->client encoding conversions on
a column-by-column basis.  (There's an assumption that metadata e.g.
column names are UCS-2.)  If it's not working, it could well be the
library doesn't recognize the server's token indicating UTF-16 encoding,
and thus doesn't assign an appropriate iconv handle.  Cf.
src/tds/token.c::tds7_get_data_info(), look for the call to
tds_iconv_from_collate().  Also src/tds/iconv.c::collate2charset(). 

You're really best off making the modifications yourself because you'll be
able to test the results.  Not many people both have 2008 and use UTF-16. 
 

> Are there any working alternatives?  

Post the chunk of the log where the metadata arrive, the data block and
the "tds7_get_data_info" paragraphs that follow.  We could patch in a line
or two blindly and see if that helps.  But you're still in the
build-and-test part of the cycle.  

HTH.

--jkl