Re: Conversion emoji symbols to UTF-8

I have tried FreeTDS from here, and all work fine. So the error on my side.
Will try to find.

On 20 April 2016 at 14:41, Velichko Yuriy <[email protected]> wrote:

>
>
> On 20 April 2016 at 14:07, Frediano Ziglio <[email protected]> wrote:
>
>> 2016-04-20 12:01 GMT+01:00 Velichko Yuriy <[email protected]>:
>> > Your query works for me to:
>> >
>> > select cast(cast(0x420435044104420420003dd84ddc20007400650073007400 as
>> > nvarchar(200)) as sql_variant)
>> >
>> > Here an example for broken result:
>> >
>> > create sequence s1;
>> > select sys_seq.start_value FROM sys.sequences sys_seq where name = 's1';
>> >
>>
>> Is this related to emoji/encoding ?
>>
> ​Already no, I think. Should I create new thread?​
>
>
>> Can you post a
>> ​​
>> TDSDUMP. I don't have a mssql 2012 to test with (it's
>> tested by AppVeyor).
>>
>> Frediano
>>
>> > On 20 April 2016 at 13:13, Frediano Ziglio <[email protected]> wrote:
>> >
>> >> 2016-04-20 9:39 GMT+01:00 Velichko Yuriy <[email protected]>:
>> >> > One more strange thing.
>> >> >
>> >> > 1. As you mentioned the default property use utf-16 differs on win
>> (TRUE)
>> >> > and mac (FALSE). So I must to set "use utf-16 = yes" to avoid
>> different
>> >> > behaviour, Good that it will be fixed in 1.0
>> >>
>> >> no, the default is the same, just that you are using the trivial iconv
>> >> provided with FreeTDS which is not recommended.
>> >>
>> >> > 2.
>> >> >
>> >> > tds version = 7.2 - has no effect for emoji
>> >> > symbols
>> >> > just need to use
>> >> > utf-16
>> >> > .
>> >> >
>> >>
>> >> Good, so now should work for every protocol version.
>> >>
>> >> > 3
>> >> >
>> >> >
>> >> >
>> >> > tds version = 7.2
>> >> > - brokes another kind of strings.
>> >>
>> >> Which ones?
>> >>
>> >> > After I set "
>> >> >
>> >> > tds version = 7.2
>> >> >
>> >> > "
>> >> > I got few broken tests.
>> >>
>> >> Which tests ?
>> >>
>> >> > I found out that strings returned for sql_variant has the same
>> problems
>> >> > like for emoji symbols - string starts with
>> >> >
>> >> > \0 symbol and has non zero length.
>> >> > Looks like similar conversion issue. Using "
>> >> >
>> >> > tds version = 7.
>> >> > 0
>> >> > "
>> >> > - fixes this issue - returned value is valid.
>> >>
>> >> Weird. I'm trying this query on tsql
>> >>
>> >> select cast(cast(0x420435044104420420003dd84ddc20007400650073007400 as
>> >> nvarchar(200)) as sql_variant)
>> >>
>> >> (which is based on your previous string) and looks the strings are
>> >> always returned correctly.
>> >> I tried from 7.0 to 7.3 without issues.
>> >>
>> >> > To test this you can create Sequence and run query:
>> >> > select sys_seq.start_value FROM sys.sequences sys_seq
>> >> >
>> >>
>> >> Can you post all queries you are using? Which exact sequence is not
>> >> working?
>> >> Are you trying some invalid coding?
>> >>
>> >> Frediano
>> >>
>> >> >
>> >> >
>> >> >
>> >> > On 19 April 2016 at 13:53, Velichko Yuriy <[email protected]>
>> >> wrote:
>> >> >
>> >> >> Yes, without "use utf-16" symbols are broken.
>> >> >> Set
>> >> >> use utf-16 = yes
>> >> >> and
>> >> >> tds version = 7.2
>> >> >> fixes this problem on mac.
>> >> >>
>> >> >> If tds version = 7.0 - symbols are broken.
>> >> >>
>> >> >> On 19 April 2016 at 13:26, Frediano Ziglio <[email protected]>
>> wrote:
>> >> >>
>> >> >>> 2016-04-18 21:46 GMT+01:00 Daniel Fazekas <[email protected]>:
>> >> >>> > It is actually odd that merely upgrading solved that emoji issue.
>> >> >>> > Emojis being largely in the Supplementary Multilingual Plane of
>> >> Unicode
>> >> >>> require true UTF-16 support rather than the legacy UCS-2 encoding
>> on
>> >> the
>> >> >>> server communication side.
>> >> >>> >
>> >> >>> > In FreeTDS terms, that means that in addition to version 0.95+
>> you
>> >> >>> ought to include the following extra setting in your configuration
>> >> file:
>> >> >>> > use utf-16 = yes
>> >> >>> >
>> >> >>> > So a complete sample configuration would look something like
>> >> >>> > tds version = 7.2
>> >> >>> > client charset = UTF-8
>> >> >>> > use utf-16 = yes
>> >> >>> >
>> >> >>> > Was it a Windows platform peculiarity that UTF-16 worked even
>> with
>> >> that
>> >> >>> required setting missing?
>> >> >>>
>> >> >>> This as replacements iconv use utf16 in place of ucs2.
>> >> >>> This however reminds me that "use utf-16" option default should be
>> >> >>> changed to 1 (true) for 1.0 version!
>> >> >>> I'm doing it right now.
>> >> >>>
>> >> >>> Frediano
>> >> >>>
>> >> >>> > In any case, I'd add that to your configuration just to be sure.
>> >> >>> > It certaintly was necessary for me on Mac and Linux.
>> >> >>> >
>> >> >>> >> On Apr 18, 2016, at 12:09, Velichko Yuriy <
>> [email protected]
>> >> >
>> >> >>> wrote:
>> >> >>> >>
>> >> >>> >> With version 0.95.91 all works fine. looks like this issue was
>> fixed
>> >> >>> >> earlier.
>> >> >>> >>
>> >> >>> >>>>>>>>> On 17 Apr 2016 12:59, "Velichko Yuriy" <
>> >> >>> [email protected]> wrote:
>> >> >>> >>>>>>>>> My .conf includes next rows:
>> >> >>> >>>>>>>>>    tds version = 7.0
>> >> >>> >>>>>>>>>    client charset = UTF-8
>> >> >>> >
>> >> >>> > _______________________________________________
>> >> >>> > FreeTDS mailing list
>> >> >>> > [email protected]
>> >> >>> > http://lists.ibiblio.org/mailman/listinfo/freetds
>> >> >>> _______________________________________________
>> >> >>> FreeTDS mailing list
>> >> >>> [email protected]
>> >> >>> http://lists.ibiblio.org/mailman/listinfo/freetds
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> С уважением, Величко Юрий.
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > С уважением, Величко Юрий.
>> >> > _______________________________________________
>> >> > FreeTDS mailing list
>> >> > [email protected]
>> >> > http://lists.ibiblio.org/mailman/listinfo/freetds
>> >> _______________________________________________
>> >> FreeTDS mailing list
>> >> [email protected]
>> >> http://lists.ibiblio.org/mailman/listinfo/freetds
>> >>
>> >
>> >
>> >
>> > --
>> > С уважением, Величко Юрий.
>> > _______________________________________________
>> > FreeTDS mailing list
>> > [email protected]
>> > http://lists.ibiblio.org/mailman/listinfo/freetds
>> _______________________________________________
>> FreeTDS mailing list
>> [email protected]
>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>
>
>
>
> --
> ​С уважением, Величко Юрий.
>



-- 
​С уважением, Величко Юрий.
_______________________________________________
FreeTDS mailing list
[email protected]
http://lists.ibiblio.org/mailman/listinfo/freetds