Hi James,
Thanks for your response. The log file indicates that the server reports the column type as UCS-2:
net.c:671:Received packet
0000 81 01 00 00 00 09 00 63-00 00 20 00 09 04 d0 00 |.......c .. .....|
0010 34 0d 00 62 00 75 00 72-00 6b 00 68 00 61 00 72 |4..b.u.r .k.h.a.r|
0020 00 64 00 5f 00 74 00 65-00 73 00 74 00 04 6e 00 |.d._.t.e .s.t..n.|
0030 61 00 6d 00 65 00 d1 10-64 75 6d 6d 79 20 74 65 |a.m.e... dummy te|
0040 78 74 70 74 72 00 00 00-64 75 6d 6d 79 54 53 00 |xtptr... dummyTS.|
0050 08 00 00 00 41 00 00 d8-d0 dd 42 00 ff 11 00 c1 |....A... ..B.....|
0060 00 01 00 00 00 79 00 00-00 00 fe 00 00 e0 00 00 |.....y.. ........|
0070 00 00 00 - |...|
token.c:510:processing result tokens. marker is 81(TDS7_RESULT)
token.c:1520:processing TDS7 result metadata.
mem.c:563:tds_free_all_results()
token.c:1544:set current_results (1 column) to tds->res_info
token.c:1555:setting up column 0
token.c:3294:adjust_character_column_size:
Server charset: UCS-2LE
Server column_size: 2097152
Client charset: UTF-8
Client column_size: 4194304
token.c:1490:tds7_get_data_info:
colname = name (4 bytes)
type = 35 (text)
server's type = 99 (UCS-2 text)
column_varint_size = 4
column_size = 4194304 (2097152 on server)
(and in the TDS documentation I don't see other ways of sending unicode characters than UCS-2). On the SQL Server type, there is no difference in the column type, whether a string contains "surrogate characters" (and needs to be interpreted as UTF-16) or not. This seems OK to me as UCS-2 is a subset of UTF-16.
Nevertheless, when I make a query in SQL Server Management Studio, it correctly converts the characters from UTF-16 to UTF-8 when writing the results to a report file.
In the example above, the value retrieved from the database is a three-UTF-16-character string: character "A", the character U+101D0 (" PHAISTOS DISC SIGN PEDESTRIAN" :-), and "B".
I have also tried using the XML data type instead of NVARCHAR, the FreeTDS log file looks similar.
From all that I learned about this, it seems it would be safe intepreting the UCS-2 characters as UTF-16, for the conversion to UTF-8 - since UTF-16 is backwards compatible to UCS-2. (see http://en.wikipedia.org/wiki/UTF-16/UCS-2 ). Maybe for now as an option to the configure script?
I would appreciate your help with such a change... for sure I can help with testing it.
Best Regards,
Burkhard
--
Neu: GMX De-Mail - Einfach wie E-Mail, sicher wie ein Brief!
Jetzt De-Mail-Adresse reservieren: http://portal.gmx.net/de/go/demail
lmpx.com only provides a reader for public news (NNTP) servers. It is not
affiliated with the servers or forums shown here and is not responsible for
the content of articles, which is written by their respective authors.