Re: FreeTDS and UTF-8 support

Sebastien,

-----Original Message-----
>From: Sebastien FLAESCH <[email protected]>
>Sent: Jul 24, 2008 7:57 AM
>To: FreeTDS Development Group <[email protected]>
>Subject: Re: [freetds] FreeTDS and UTF-8 support
>
>James K. Lowden wrote:
>> Sebastien FLAESCH wrote:
>>> I think FreeTDS could support all single-byte or multi-byte character
>>> sets if:
>>>
>>> 1) The application (main) calls setlocale() to define the current
>>> charset for libc functions like mbtowc, wctomb, mblen (that would
>>> include UTF-8 support).
>> 
>> I think we are nearly already there.  We use setlocale(3) to determine the
>> client encoding by default, and consult the freetds.conf "client charset"
>> property as an override.  
>
>Cool.
>
>> Instead of the "mb" functions, we use iconv(3).  We do sometimes depend on
>> strlen(3), which is one of many reasons that UCS-2 is not supported (yet)
>> as a client encoding.  (Another reason is that the APIs sometimes assume
>> bytes == characters, or state that they return a pointer to a
>> null-terminated string.  Another reason is I'm aware of only two operating
>> systems that use any form of wchar_t natively, and they're both
>> proprietary.) 
>> 
>>> SQLPrepare(hp, (char *) "my SQL string in current SBCD or MBCS charset",
>>> SQL_NTS);
>>>
>>> would be mapped to:
>>>
>>> SQLPrepareW(hp, (wchar_t *) "my SQL string in UCS-2", SQL_NTS);
>> 
>> I don't think SQLPrepareW is necessary.  SQLPrepare could do both jobs. 
>
>Oh oh take care, the ODBC headers of UnixODBC define SQLPrepareW as follows:
>
>SQLRETURN SQL_API SQLPrepareW(
>     SQLHSTMT           hstmt,
>     SQLWCHAR        *szSqlStr,
>     SQLINTEGER         cbSqlStr);
>
>while SQLPrepare is defined as:
>
>SQLRETURN SQL_API SQLPrepareA(
>     SQLHSTMT           hstmt,
>     SQLCHAR             *szSqlStr,
>     SQLINTEGER         cbSqlStr);
>
>SQLWCHAR is #define UNICODE dependent, and I think it can be wchar_t,
>while SQLCHAR can only be char.

SQLWCHAR _is_ #define UNICODE dependent (that's why it has W in it's name),
however it is resolving itself to be unsigned short. When I was looking at
the unixODBC headers, I was also confused and had to ask the maintainer.

The reason being (according to him) is that 'unsigned short' is 2 bytes
everywhere, and also because SQLWCHAR is defined as UTF-16 on Windows.
I don't know about SQLCHAR though. My program is UNICODE only.

>
>Don't know how you can deal with this without an explicit cast (char *) but that's
>really not a good idea IMHO.
>
>Anyway, I am not sure SQLPrepareW() / wchar_t is really an immediate need on UNIX.
>
>So if you support SQLPrepare() for single-byte and multi-byte charsets it's great.
>
>....
>
>FYI:
>
>To share my experience with Microsoft Native Client ODBC programming in WideChar,
>concerning SQL parameter binding and column description:
>
>It appears that the "LengthPrecision" parameter of SQLBindParameter() must be passed
>as a number of characters, not a number of bytes, while the size of the data buffer
>(StrLen_or_Ind) has to be specified in bytes. Can be confusing...
>
>Fortunately, when using SQLDescribeCol(), the ColumnSize output param holds a number
>of characters, not bytes.
>
>Cheers
>Seb
>
>>> WARNING: wchar_t on UNIX is 4 bytes while UCS-2 and wchar_t on Windonws
>>> is 2 bytes.
>> 
>> Thanks for that.  I didn't know.  
>> 
>> Regards, 
>> 
>> --jkl

Thank you.

>> _______________________________________________
>> FreeTDS mailing list
>> [email protected]
>> http://lists.ibiblio.org/mailman/listinfo/freetds
>> 
>
>_______________________________________________
>FreeTDS mailing list
>[email protected]
>http://lists.ibiblio.org/mailman/listinfo/freetds