Re: Force use of UTF-8 instead of ISO8859-1

Frediano Ziglio wrote:
> >> correctly our imlpementation returns
> >> 10 and.... (rumble!) SQL_WCHAR... I forget this SMALL detail....
> >
> > What should it return if the encoding is UTF-8?  SQL_WCHAR seems
> > correct, unless it means UCS-2LE.  I think SQL_WCHAR means "Unicode".
> >
> 
> Not only this, it also specify an encoding. For instance if you use
> unixODBC is UCS-2 or system wchar_t using iODBC.
...
> > SQL_DESC_OCTET_LENGTH is the length in bytes of the buffer needed hold
> > the data.  The driver knows how the buffer is encoded.  It should
> > return -- as dbcollen() does -- the maximum size that could be
> > required to hold any value that the column could hold.  For nchar(10)
> > in UTF-8, that's 40.
> >
> 
> True and false I think... octet length is the buffer size using
> default client type for a given server type.... 20 for a
> nchar(10) if client have sizeof(SQLWCHAR) == 2 cause output is
> supposed to be a "wide" character set (UCS-2/UCS-4... or sometimes
> UTF-16... never UTF-8). Client character set affect how we translate
> to multi-byte character set but not how library encodes wide character
> set.

Wouldn't it be 40 for USC-4? 

> Client character set affect how we translate
> to multi-byte character set but not how library encodes wide character
> set. 

No, "how we translate" === "how library encodes".  Character data have
only two forms:  server encoding and client encoding.  The library has no
encoding.  All data that the client exchanges with the library always uses
one encoding: the client's.  

The DM does not know the encoding FreeTDS is using, but the application
does.  The application is asking SQL_DESC_OCTET_LENGTH for the size of the
buffer it needs to allocate.  FreeTDS has already converted UCS-2 to
utf-8.  It should return 40, else the application is likely to allocate
too small a buffer.  

Should SQLDescribeCol return SQL_WCHAR for utf-8 data?  I think Yes.  If
it returned a new constant e.g. SQL_U8CHAR, most applcations wouldn't know
how to deal with it.  If it returns SQL_CHAR, the application might assume
1 byte/char.  SQL_WCHAR is the best option.  If the application assumes
SQL_WCHAR means UCS-2, then the client should use UCS-2 encoding instead. 


I think we basically agree here.  You see the changes to bsqlodbc, which
now works.  I don't see another way for the application to determine the
buffer size.  

Regards, 

--jkl