Re: Status of UTF-8 support in 0.83 dev

Hello James,

James K. Lowden wrote:
> Hi Sebastien, 
> 
>> When I print the client and the server charset in tds_convert_string()
>> called from tds_put_data(), I see:
>>
>>    char_conv->client_charset.name = "UTF-8"
>>    char_conv->server_charset.name = "CP1252"
>>
>> I expected UCS-2 for the server charset...????
>>
>> Hence it's normal that the conversion of non-CP1252 compatible chars
>> like é´» result in error...
> 
> That's OK.  I think you misunderstand the meaning of
> client2server_chardata: "server_chardata" is the server's single-byte
> encoding, used for char/varchar/text columns.  The server announces its
> encoding as a message immediately after logon.  
> 
> (Off the top of my head, I can't think why we care about the server's
> single-byte encoding.  Maybe for TDS 4.2.  TDS 7+ uses UCS-2; we never
> send the server anything else IIRC.) 
> 
> As I said yesterday, CP1252 could explain your 2-byte/3-byte observation. 
> 
> 
> First assertion: in unprefixed strings, *only* characters represented by
> CP1252 will be stored correctly on your server.  All others will be
> converted to CP1252 first by the server, with varying results.  Pick *any*
> non-CP1252 character and see what happens.  
> 
> Second assertion: all prefixed strings embedded in SQL text will be
> handled correctly on both ends.  
> 
> Final assertion: parameterized query buffers are not correctly converted
> when sent to the server.  
> 
> That's where I think we are.  Agreed?  

Yes, I think so...

And as I wrote in a previous mail, we should clarify how UTF-8 buffers of
parameterized queries have to be bound...

Should ParameterType be SQL_[VAR]CHAR or SQL_W[VAR]CHAR ?

Looking at the code, client2ucs2 TDSICONV is only used if SQL_W[VAR]CHAR
is used, but that does not fit the IBM DB2 CLI way and confuses me since
SQL_W[VAR]CHAR is for me the MS "Wide Char Unicode", not UTF-8...

The fact that I see unexpected chars at the end of the UTF-8 string in
tds_put_data (from curcol->column_data), is probably because the string
is not null terminated at the same place as my original string (just
guessing).

     UTF-8 = [abcéô鴻01000]  ( size= 10 )

...

Thanks for you answer, I really appreciate your help.



> 
> Regards, 
> 
> --jkl
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds
> 

_______________________________________________
FreeTDS mailing list
[email protected]
http://lists.ibiblio.org/mailman/listinfo/freetds