Re: SQLGetData() returns TEXT length minus one on Solaris 64b with SQL Server 2005

It appears that compiling with libiconv does not help.
Seb


On 02/05/2013 12:46 PM, Sebastien FLAESCH wrote:
> On 02/04/2013 06:57 PM, Frediano Ziglio wrote:
>> 2013/2/4 Sebastien FLAESCH<[email protected]>:
>>> Frediano,
>>>
>>> I believe it's related to the client charset conversions...
>>>
>>> I did following simple test:
>>>
>>> My database collation is CP1252, so to avoid iconv usage, in the ODBC
>>> data source definition, I have now:
>>>
>>> ClientCharset = CP1252
>>>
>>> The program uses only pure ASCII chars, so this is ok.
>>>
>>> Now there is no more iconv error in the log, and the first call to
>>> SQLGetData() returns 10000 bytes as expected...
>>>
>>> If fact for character data, ODBC returns the char pieces with an
>>> additional \0 terminator.
>>>
>>> So, with a buffer of 4096 bytes:
>>>
>>> Call #1 to SQLGetData() returns len=10000 bytes (ok)
>>> Call #2 to SQLGetData() returns len=5905 bytes
>>>      there was 5905 bytes left to read, because only 4095 bytes could
>>>      be written to the buffer, because of additional \0 terminator.
>>>      10000 - 4095 = 5905 ...
>>> Call #3 to SQLGetData() returns len=1810 bytes
>>>       5905 - 4095 = 1810 ...
>>> Last call: 1810 bytes fit in the 4096 buffer.
>>>
>>
>> Well, this is expected. But I don't understand what's going wrong with Solaris.
>> Could be possible that Solaris do not convert some characters and so
>> it convert a byte characters to 0 bytes ??
>
> Possible...
>
>> Are you sure that there is only ASCII characters (0-127) ??
>
> Pure ASCII.
>
>> Why should Solaris report some warning converting ASCII ??
>
> My C locale is set to POSIX/C, I don't know why it ends as ISO-8859-1
> for the iconv conversions in FreeTDS. Maybe it's the default.
>
> Is there some mapping between the C locale (LANG/LC_ALL) and the iconv
> locale name (I know these can be different names, how is this mapped?)
>
>> A dump would be helpful. Send privately to me if it contains sensitive
>> information.
>
> I send you a dump.
>
>> You could try using libiconv.
>
> We'll try this too.
>
>>> I hope this helps...
>>>
>>> Seb
>>>
>>
>> Frediano
>>
>>
>>>
>>> On 02/03/2013 12:12 PM, Frediano Ziglio wrote:
>>>> Hi,
>>>>      where is SQL_DATA_AT_EXEC in your code?
>>>>
>>>> Which DM are you using? Is possible that for some reason DM call some
>>>> additional function to do some checks. This could discovered enabling
>>>> FreeTDS dump.
>>>>
>>>> Quire strange it works correctly on Linux but not on Solaris.
>>>> Does data contains no-ASCII characters?
>>>> How long are usually these TEXT fields?
>>>> Which TDS protocol version are you using?
>>>> There is a specific getdata test for this stuff and should work.
>>>>
>>>> Regards,
>>>>      Frediano
>>>>
>>>> 2013/2/2 Sebastien FLAESCH<[email protected]>:
>>>>> Hi Frediano,
>>>>>
>>>>> FYI, we use a single-byte encoding (LANG=POSIX) ...
>>>>>
>>>>> Here is the code we use, nothing particular, this is the same code we
>>>>> use for different ODBC drivers...
>>>>>
>>>>> #define BLOB_BLOCK_SIZE 4096
>>>>> ....
>>>>> static SQLRETURN getBlobPieces(SqlStatement * st, SQLUSMALLINT fpos,
>>>>>                                    SQLSMALLINT ctype, MyLocator * loc,
>>>>>                                    FILE * out, SQLLEN * plen)
>>>>> {
>>>>>         SQLRETURN r = 0;
>>>>>         char buf[BLOB_BLOCK_SIZE];
>>>>>         SQLLEN len;
>>>>>         int i = 0;
>>>>>         while (1) {
>>>>>             r = SQLGetData(st->stmtHandle, fpos, ctype,
>>>>>                            (SQLPOINTER) buf, sizeof(buf),&len);
>>>>>             if (!SQL_SUCCEEDED(r) || len == 0 || len == SQL_NULL_DATA)
>>>>>                 break;
>>>>>             if (len>    (SQLLEN) sizeof(buf) || len == SQL_NO_TOTAL)
>>>>>                 len = sizeof(buf);
>>>>>             /* Char data piece get \0, these must be removed! */
>>>>>             if (ctype == SQL_C_CHAR) {
>>>>>                 if (buf[len - 1] == '\0') {
>>>>>                     len--;
>>>>>                 }
>>>>>             }
>>>>>             fwrite(buf, 1, len, out);
>>>>>         }
>>>>>         *plen = len;
>>>>>         return r;
>>>>> }
>>>>>
>>>>> Am I missusing SQLGetData()? As you can see, I modify the len after
>>>>> Does this affect the next call to SQLGetData()?
>>>>>
>>>>> Anyway, the INITIAL len returned by SQLGetData() is for sure:
>>>>>
>>>>>        (total-bytes - 1)
>>>>>
>>>>> When using UTF-8, I get SQL_NO_TOTAL in the first call, but in ASCII,
>>>>> I get the (total size - 1)...
>>>>>
>>>>> Seb
>>>>>
>>>>> On 02/02/2013 08:52 AM, Frediano Ziglio wrote:
>>>>>> Il giorno 31/gen/2013, alle ore 14:28, Sebastien FLAESCH<[email protected]>     ha scritto:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Context:
>>>>>>>
>>>>>>> $ uname -a
>>>>>>> SunOS s64x100 5.10 Generic_120012-14 i86pc i386 i86pc
>>>>>>>
>>>>>>> FreeTDS versions tested:
>>>>>>> - freetds-0.92.dev.20120312
>>>>>>> - freetds-dev.0.92.377
>>>>>>>
>>>>>>> When fetching a TEXT from the database (SQL Server 2005), using SQL_DATA_AT_EXEC,
>>>>>>> it appears that the SQLGetData() function returns the size of the text data,
>>>>>>> minus one...
>>>>>>>
>>>>>>> I have tested the same FreeTDS version on a Linux 32b, and SQLGetData() returns
>>>>>>> the correct length as expected...
>>>>>>>
>>>>>>
>>>>>> Strange! How do you read the length?
>>>>>>
>>>>>>> I will continue to investigate, but I wanted to ask if this is a know issue.
>>>>>>>
>>>>>>> When connecting to SQL Server 2008 R2, this does not occur, because SQLGetData()
>>>>>>> returns SQL_NO_TOTAL and we fetch text pieces in a different way...
>>>>>>>
>>>>>>
>>>>>> This is done to deal with different encodings.
>>>>>>
>>>>>>> Thanks
>>>>>>> Seb
>>>>>>
>>>>>> Frediano
>>>>>>
>>>>>> _______________________________________________
>>>>>> FreeTDS mailing list
>>>>>> [email protected]
>>>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeTDS mailing list
>>>>> [email protected]
>>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>> _______________________________________________
>>>> FreeTDS mailing list
>>>> [email protected]
>>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>>>
>>>
>>> _______________________________________________
>>> FreeTDS mailing list
>>> [email protected]
>>> http://lists.ibiblio.org/mailman/listinfo/freetds
>> _______________________________________________
>> FreeTDS mailing list
>> [email protected]
>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>
>
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds
>