Re: Bulk upload problem

Hi Bill,

thanks for taking a look at this problem. We figured out the problem and
have made a patch and used it locally.
The bug is in blk_rowxfer_mult (in src/ctlib/blk.c), that inspite of passing
in the rowcount as the number of rows from the datafmt structure in
blk_bind, it was being set to 1 in the function (not sure why ?)


CS_RETCODE blk_rowxfer(CS_BLKDESC * blkdesc)

{

*CS_INT row_count = 1;*

*//return blk_rowxfer_mult(blkdesc, &row_count); //problem*

return blk_rowxfer_mult(blkdesc, NULL); // our fix

}

You are right in that the freeTDS code is inconsistant with the sybase
manual for this particular function, since the manual says blk_rowxfer_mult
with NULL rowcount should be equivalent to blk_rowxfer, which is clearly not
the case here.
BTW, when you say this is based on a deprecated version of sybase ctlib, do
you mean there is a better alternative for transparent connectivity to
sybase & / or SQL Server?

Thanks and regards,
Shatam




On Wed, Jul 14, 2010 at 7:11 PM, Thompson, Bill D (London) <
[email protected]> wrote:

>  Hi Shatam,
>
> I'll happily take a look at this for you, but first I'd need a log file
> that I can read.
> Your attached file (freetds_snippet.txt) doesn't look like a freetds log
> file to me, and is full of unreadable characters.
> Was this produced using the TDSDUMP functionality ?
>
> In the meantime, I can see you make a call to blk_rowxfer()
>
> The SYBASE manuals I can see online state that blk_rowxfer() can be
> called to insert many rows with one call, the application previously
> having called blk_bind() appropriately to indicate that it is binding an
> array of variables.
>
> However, the freetds code is explicit - blk_rowxfer() will only insert
> one row.
>
> The ct-library bulk loading functions were written (by me) a LONG time
> ago, and were developed based on a long deprecated version of Sybase and
> its client libraries. I can only assume that the capabilities of
> blk_rowxfer() have been enhanced in the meantime.
> The alternative to that hypothesis is that I didn't read the manual
> pages properly, or wilfully ignored them. Which is of course possible.
>
> However, to try and help... our version of the bulk library has two
> functions:
>
> CS_RETCODE blk_rowxfer(CS_BLKDESC * blkdesc)
> CS_RETCODE blk_rowxfer_mult(CS_BLKDESC * blkdesc, CS_INT * row_count)
>
> The latter function *should* do the multiple row inserts, as it's name
> suggests.
> You just have to pass it additionally a count of the rows you wish to
> insert.
>
> I hope this helps,
>
> Bill
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of shatam
> bhattacharya
> Sent: 10 July 2010 11:11
> To: [email protected]
> Subject: [freetds] Bulk upload problem
>
>
>  Hi Gurus,
>
> This might have been discussed before, but googling did not help. So am
> posting this question.
>
> Recently our employer decided to move all strategic systems to MS Sql
> Server
> from Sybase and for this we opted for freeTDS as the underlying
> connectivity
> platform. We are using it for perl and c++ applications. While the perl
> modules are working fine, one of the c++ apps has this wierd problem in
> bulk
> upload. The problem is when we try to do a bulk upload it only inserts
> the
> first row of the recordset. We have seen this both with temp table and
> physical tables. Note that this problem is happening both in Sybase and
> MS
> Sql Server uisng freeTDS libs.
>
> The same c++ app works fine with the sybclient library on linux.
> Attached is
> a sample log file dumped by freeTDS (the relevent db operations are
> around
> the end of the log).
>
> Is there a flag / option to tweek this and make it work. Please let me
> know
> if you need any more detials about the application / library level
> connectivity code / logs.
>
> Any help is greatly appreciated.
>
>
> Following is the code snippet to connect -
> if( _ctLibShim->blk_init( _bulkDescriptor, CS_BLK_IN, const_cast< char*
> >(
> _tableName.c_str() ), _tableName.length() ) != CS_SUCCEED )
>   {
>       throwReinitialisationError( "Failed to initialise the bulk
> descriptor
> for table " + _tableName  );
>   }
>   const int numberOfBindings( bindings_.size() );
>
>   ScopeGuard cancelBulkOperation( boost::bind(
> &InternalBulkLoad::cancelBulkOperation, this ), _logger );
>  * for( int bindingIndex = 0; bindingIndex < numberOfBindings;
> ++bindingIndex )
>   {
>       bind( *bindings_[ bindingIndex ], bindingIndex + 1 );
>   }*
> *       if( _ctLibShim->blk_rowxfer( _bulkDescriptor ) != CS_SUCCEED )
>   {
>       throwReinitialisationError( "Failed to transfer the data into
> table "
> + _tableName );
>   }*
> *   if( _ctLibShim->blk_done( _bulkDescriptor, CS_BLK_ALL, NULL ) !=
> CS_SUCCEED )
>   {
>       throwReinitialisationError( "Failed to commit the batch while
> loading
> data into table " + _tableName );
>   }*
> *  *     /*if( _ctLibShim->blk_drop( _bulkDescriptor ) != CS_SUCCEED )
>   {
>       throwReinitialisationError( "Failed to drop bulk descriptor while
> loading data into table " + _tableName );
>   }*/
>   cancelBulkOperation.release();
>
> --
> Thanks and Regards,
> Shatam Bhattacharya.
>
> --------------------------------------------------------------------------
> This message w/attachments (message) may be privileged, confidential or
> proprietary, and if you are not an intended recipient, please notify the
> sender, do not use or share it and delete it. The information contained in
> this e-mail was obtained from sources believed to be reliable; however, the
> accuracy or completeness of this information is not guaranteed. Unless
> specifically indicated, this message is not an offer to sell or a
> solicitation of any investment products or other financial product or
> service, an official confirmation of any transaction, or an official
> statement of Merrill Lynch. Subject to applicable law, Merrill Lynch may
> monitor, review and retain e-communications (EC) traveling through its
> networks/systems. The laws of the country of each sender/recipient may
> impact the handling of EC, and EC may be archived, supervised and produced
> in countries other than the country in which you are located. This message
> cannot be guaranteed to be secure or error-free. Refe
>  rences to "Merrill Lynch" are references to any company in the Merrill
> Lynch & Co., Inc. group of companies, which are wholly-owned by Bank of
> America Corporation. Securities and Insurance Products: * Are Not FDIC
> Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit
> * Are Not a Condition to Any Banking Service or Activity * Are Not Insured
> by Any Federal Government Agency. Past performance is no guarantee of future
> results. Attachments that are part of this E-communication may have
> additional important disclosures and disclaimers, which you should read.
> This message is subject to terms available at the following link:
> http://www.ml.com/e-communications_terms/. By messaging with Merrill Lynch
> you consent to the foregoing.
> --------------------------------------------------------------------------
>
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds
>



-- 
Thanks and Regards,
Shatam Bhattacharya.