Re: FreeTDS + SQL Server 2012 + UTF-16LE

On Fri, Aug 29, 2014 at 9:56 AM, Frediano Ziglio <[email protected]> wrote:

> 2014-08-29 17:50 GMT+01:00 John Anderson <[email protected]>:
>
> > Hello!  I'm wondering if you had any tips on how to work with characters
> > larger than 0xFFFF?  We currently have the problem where we try to
> > insert/update a row in the database and it gets truncated at the
> character
> > and creates an invalid syntax:
> >
> > For example this character:   \U0001f44d
> >
> > We've tracked this down to the fact that FreeTDS is trying to convert
> this
> > to UCS-2 rather than UTF16-LE, which is what SQL Server 2012 uses.
> >
> > For example if we say:
> >
> > INSERT INTO table(name) VALUES(N' Hello \U0001f44d')
> >
> >
> How are you using this string?
>
>
Sorry, this is the python representation of the string which is a "Thumbs
Up", It's UTF-8 hex representation is 0xf09f918d and what needs to be
stored in the database is UTF-16LE hex 0x3dd84ddc.

This is the character:

http://www.fileformat.info/info/unicode/char/1F44D/index.htm

But it is any character above 0xFFFF will cause the issue.

So the actually query ends up looking like N'Hello <display of a thumbs up>'



>
> > We get the error:
> >
> > Unclosed quotation mark after the character string 'Hello ' because it
> > stops processing the rest of the query.
> >
> > Is there a freetds.conf or environment setting we can use to tell it to
> use
> > UTF-16LE instead of UCS2 so that we can accept these larger character
> sets?
> >
> > As a temporary solution we've started sending the raw UTF-16LE bytes to
> > FreeTDS but we prefer not having to do this.
> >
> > Thanks,
> > John
> >
>
> Frediano
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds
>