Re: FreeTDS + SQL Server 2012 + UTF-16LE
Marc Abramowitz <[email protected]>
| Newsgroups | gmane.comp.db.tds.freetds |
|---|---|
| Message-ID | <[email protected]> |
Interesting. John, I wonder if you can write a little C program that illustrates the bug? I think that the FreeTDS authors are most comfortable in C, so this will probably illustrate the problem best for them and show beyond a shadow of a doubt that there's nothing happening in pymssql. -Marc http://marc-abramowitz.com Sent from my iPhone 4S > On Aug 29, 2014, at 7:07 AM, John Anderson <[email protected]> wrote: > >> On Fri, Aug 29, 2014 at 9:56 AM, Frediano Ziglio <[email protected]> wrote: >> >> 2014-08-29 17:50 GMT+01:00 John Anderson <[email protected]>: >> >>> Hello! I'm wondering if you had any tips on how to work with characters >>> larger than 0xFFFF? We currently have the problem where we try to >>> insert/update a row in the database and it gets truncated at the >> character >>> and creates an invalid syntax: >>> >>> For example this character: \U0001f44d >>> >>> We've tracked this down to the fact that FreeTDS is trying to convert >> this >>> to UCS-2 rather than UTF16-LE, which is what SQL Server 2012 uses. >>> >>> For example if we say: >>> >>> INSERT INTO table(name) VALUES(N' Hello \U0001f44d') >> How are you using this string? > Sorry, this is the python representation of the string which is a "Thumbs > Up", It's UTF-8 hex representation is 0xf09f918d and what needs to be > stored in the database is UTF-16LE hex 0x3dd84ddc. > > This is the character: > > http://www.fileformat.info/info/unicode/char/1F44D/index.htm > > But it is any character above 0xFFFF will cause the issue. > > So the actually query ends up looking like N'Hello <display of a thumbs up>' > > > >> >>> We get the error: >>> >>> Unclosed quotation mark after the character string 'Hello ' because it >>> stops processing the rest of the query. >>> >>> Is there a freetds.conf or environment setting we can use to tell it to >> use >>> UTF-16LE instead of UCS2 so that we can accept these larger character >> sets? >>> >>> As a temporary solution we've started sending the raw UTF-16LE bytes to >>> FreeTDS but we prefer not having to do this. >>> >>> Thanks, >>> John >> >> Frediano >> _______________________________________________ >> FreeTDS mailing list >> [email protected] >> http://lists.ibiblio.org/mailman/listinfo/freetds > _______________________________________________ > FreeTDS mailing list > [email protected] > http://lists.ibiblio.org/mailman/listinfo/freetds