Re: Bulk upload problem

Applied to CVS

bye
  freddy77

2010/7/14 shatam bhattacharya <[email protected]>:
> Hi Bill,
>
> thanks for taking a look at this problem. We figured out the problem and
> have made a patch and used it locally.
> The bug is in blk_rowxfer_mult (in src/ctlib/blk.c), that inspite of passing
> in the rowcount as the number of rows from the datafmt structure in
> blk_bind, it was being set to 1 in the function (not sure why ?)
>
>
> CS_RETCODE blk_rowxfer(CS_BLKDESC * blkdesc)
>
> {
>
> *CS_INT row_count = 1;*
>
> *//return blk_rowxfer_mult(blkdesc, &row_count); //problem*
>
> return blk_rowxfer_mult(blkdesc, NULL); // our fix
>
> }
>
> You are right in that the freeTDS code is inconsistant with the sybase
> manual for this particular function, since the manual says blk_rowxfer_mult
> with NULL rowcount should be equivalent to blk_rowxfer, which is clearly not
> the case here.
> BTW, when you say this is based on a deprecated version of sybase ctlib, do
> you mean there is a better alternative for transparent connectivity to
> sybase & / or SQL Server?
>
> Thanks and regards,
> Shatam
>
>
>
>
> On Wed, Jul 14, 2010 at 7:11 PM, Thompson, Bill D (London) <
> [email protected]> wrote:
>
>>  Hi Shatam,
>>
>> I'll happily take a look at this for you, but first I'd need a log file
>> that I can read.
>> Your attached file (freetds_snippet.txt) doesn't look like a freetds log
>> file to me, and is full of unreadable characters.
>> Was this produced using the TDSDUMP functionality ?
>>
>> In the meantime, I can see you make a call to blk_rowxfer()
>>
>> The SYBASE manuals I can see online state that blk_rowxfer() can be
>> called to insert many rows with one call, the application previously
>> having called blk_bind() appropriately to indicate that it is binding an
>> array of variables.
>>
>> However, the freetds code is explicit - blk_rowxfer() will only insert
>> one row.
>>
>> The ct-library bulk loading functions were written (by me) a LONG time
>> ago, and were developed based on a long deprecated version of Sybase and
>> its client libraries. I can only assume that the capabilities of
>> blk_rowxfer() have been enhanced in the meantime.
>> The alternative to that hypothesis is that I didn't read the manual
>> pages properly, or wilfully ignored them. Which is of course possible.
>>
>> However, to try and help... our version of the bulk library has two
>> functions:
>>
>> CS_RETCODE blk_rowxfer(CS_BLKDESC * blkdesc)
>> CS_RETCODE blk_rowxfer_mult(CS_BLKDESC * blkdesc, CS_INT * row_count)
>>
>> The latter function *should* do the multiple row inserts, as it's name
>> suggests.
>> You just have to pass it additionally a count of the rows you wish to
>> insert.
>>
>> I hope this helps,
>>
>> Bill
>>
>> -----Original Message-----
>> From: [email protected]
>> [mailto:[email protected]] On Behalf Of shatam
>> bhattacharya
>> Sent: 10 July 2010 11:11
>> To: [email protected]
>> Subject: [freetds] Bulk upload problem
>>
>>
>>  Hi Gurus,
>>
>> This might have been discussed before, but googling did not help. So am
>> posting this question.
>>
>> Recently our employer decided to move all strategic systems to MS Sql
>> Server
>> from Sybase and for this we opted for freeTDS as the underlying
>> connectivity
>> platform. We are using it for perl and c++ applications. While the perl
>> modules are working fine, one of the c++ apps has this wierd problem in
>> bulk
>> upload. The problem is when we try to do a bulk upload it only inserts
>> the
>> first row of the recordset. We have seen this both with temp table and
>> physical tables. Note that this problem is happening both in Sybase and
>> MS
>> Sql Server uisng freeTDS libs.
>>
>> The same c++ app works fine with the sybclient library on linux.
>> Attached is
>> a sample log file dumped by freeTDS (the relevent db operations are
>> around
>> the end of the log).
>>
>> Is there a flag / option to tweek this and make it work. Please let me
>> know
>> if you need any more detials about the application / library level
>> connectivity code / logs.
>>
>> Any help is greatly appreciated.
>>
>>
>> Following is the code snippet to connect -
>> if( _ctLibShim->blk_init( _bulkDescriptor, CS_BLK_IN, const_cast< char*
>> >(
>> _tableName.c_str() ), _tableName.length() ) != CS_SUCCEED )
>>   {
>>       throwReinitialisationError( "Failed to initialise the bulk
>> descriptor
>> for table " + _tableName  );
>>   }
>>   const int numberOfBindings( bindings_.size() );
>>
>>   ScopeGuard cancelBulkOperation( boost::bind(
>> &InternalBulkLoad::cancelBulkOperation, this ), _logger );
>>  * for( int bindingIndex = 0; bindingIndex < numberOfBindings;
>> ++bindingIndex )
>>   {
>>       bind( *bindings_[ bindingIndex ], bindingIndex + 1 );
>>   }*
>> *       if( _ctLibShim->blk_rowxfer( _bulkDescriptor ) != CS_SUCCEED )
>>   {
>>       throwReinitialisationError( "Failed to transfer the data into
>> table "
>> + _tableName );
>>   }*
>> *   if( _ctLibShim->blk_done( _bulkDescriptor, CS_BLK_ALL, NULL ) !=
>> CS_SUCCEED )
>>   {
>>       throwReinitialisationError( "Failed to commit the batch while
>> loading
>> data into table " + _tableName );
>>   }*
>> *  *     /*if( _ctLibShim->blk_drop( _bulkDescriptor ) != CS_SUCCEED )
>>   {
>>       throwReinitialisationError( "Failed to drop bulk descriptor while
>> loading data into table " + _tableName );
>>   }*/
>>   cancelBulkOperation.release();
>>
>> --
>> Thanks and Regards,
>> Shatam Bhattacharya.
>>
_______________________________________________
FreeTDS mailing list
[email protected]
http://lists.ibiblio.org/mailman/listinfo/freetds