Re: FreeTDS Crash: free(): invalid next size (fast)

I have updated the pyodbc issue:

> Sorry for all the comments, I am continuing to try to figure this one out.
>
> I was able to prevent my application's unit tests from crashing by changing all my SQLAlchemy columns from String() to Unicode().  That lead me to be able to come up with a test script that reproduces the problem (attached) based on SQLAlchemy.
>
> In my attempt to translate that code to pure pyodbc (the test script previously attached), the error no longer shows up.  However, I'm pretty confident the bug is in pyodbc or freetds since pure python code shouldn't be able to crash python.
>
> FreeTDS guys suggested I run valgrind.  So I did that:
>
> valgrind --tool=memcheck -v --suppressions=valgrind-python.supp python test_freetds_unicode.py 2>valgrind-out.txt
>
> based on the files here:
>
> http://svn.python.org/projects/python/trunk/Misc/README.valgrind
> http://svn.python.org/projects/python/trunk/Misc/valgrind-python.supp  (I uncommented the parts they recommend uncommenting if you don't want to re-compile python)
>
> However, when I run with valgrind, I don't get a segfault anymore!  Valgrind output is attached anyway, not sure how helpful it will be.
>
> Let me know if there is anything else I can to help.
So it does seem to be related to encoding issues, but the crash 
disappears when I use valgrind, so I am not sure where to go at this point.

--------------------------------------
Randy Syring
Intelicom
Direct: 502-276-0459
Office: 502-212-9913

For the wages of sin is death, but the
free gift of God is eternal life in
Christ Jesus our Lord (Rom 6:23)


On 04/08/2011 10:58 AM, Frediano Ziglio wrote:
> valgrind is your friend :)
>
> I bet is related to a conflict of SQLWCHAR ABI
>
> freddy77
>
> 2011/4/8 Randy Syring<[email protected]>:
>> FYI, one of the original links I posted was wrong, the correct one is:
>>
>> http://rcs-comp.com/tmp/freetds-082-1build6.log
>>
>> But, I am kind of stumped at this point.  The simple test case filed on
>> the pyodbc issue was "fixed" by using a CVS version of FreeTDS and
>> ClientCharset=UTF-8.  But, I am still getting crashes when running my
>> unit tests for my web application with SQLAlchemy + pyodbc.
>>
>> The last part of the dump file, before the crash, is:
>>
>> token.c:554:processing result tokens.  marker is  ff(DONEINPROC)
>> token.c:2335:tds_process_end: more_results = 1
>>          was_cancelled = 0
>>          error = 0
>>          done_count_valid = 1
>> token.c:2366:                rows_affected = 1
>> util.c:156:Changed query state from READING to PENDING
>> odbc.c:3534:odbc_process_tokens: tds_process_tokens returned 1
>> odbc.c:3535:    result_type=4054, TDS_DONE_COUNT=10, TDS_DONE_ERROR=0
>> odbc.c:3599:odbc_process_tokens: processed TDS_DONEINPROC_RESULT
>> odbc.c:3374:_SQLExecute: odbc_process_tokens returned result_type 4054
>> token.c:539:tds_process_tokens(0x36fffa0, 0x7fff573205ac,
>> 0x7fff573205a8, 0x6914)
>> util.c:156:Changed query state from PENDING to READING
>> token.c:554:processing result tokens.  marker is  79(RETURNSTATUS)
>> token.c:554:processing result tokens.  marker is  ac(PARAM)
>> token.c:611:processing parameters for sp 13
>> token.c:613:calling tds_process_param_result
>> token.c:1243:tds_process_param_result(0x36fffa0, 0x7fff57320528)
>> token.c:1587:tds_get_data_info(0x36fffa0, 0x3a16b70, 1) [for parameter]
>> token.c:1650:processing result. type = 38(integer-null), varint_size 1
>> token.c:1655:processing result. column_size 4
>> token.c:2045:tds_get_data: type 38, varint size 1
>> token.c:2106:tds_get_data(): wire column size is 4
>> log.c:493:type integer-null has value 65
>> token.c:617:1 hidden return parameters
>> token.c:554:processing result tokens.  marker is  fe(DONEPROC)
>> token.c:2335:tds_process_end: more_results = 0
>>          was_cancelled = 0
>>          error = 0
>>          done_count_valid = 0
>> token.c:2351:tds_process_end() state set to TDS_IDLE
>> util.c:156:Changed query state from READING to IDLE
>> token.c:2366:                rows_affected = 0
>> util.c:104:logic error: cannot change query state from IDLE to PENDING
>> odbc.c:3534:odbc_process_tokens: tds_process_tokens returned 1
>> odbc.c:3535:    result_type=4053, TDS_DONE_COUNT=0, TDS_DONE_ERROR=0
>> odbc.c:3578:odbc_process_tokens: processed TDS_DONEPROC_RESULT
>> token.c:539:tds_process_tokens(0x36fffa0, 0x7fff573205ac,
>> 0x7fff573205a8, 0x26914)
>> token.c:542:tds_process_tokens() state is COMPLETED
>> odbc.c:3534:odbc_process_tokens: tds_process_tokens returned 2
>> odbc.c:3535:    result_type=4052, TDS_DONE_COUNT=0, TDS_DONE_ERROR=0
>> odbc.c:3374:_SQLExecute: odbc_process_tokens returned result_type 4046
>> odbc.c:3519:SQLExecute returns SQL_SUCCESS
>> odbc.c:4154:SQLFreeStmt(0x381e960, 3)
>> odbc.c:4075:_SQLFreeStmt(0x381e960, 3, 0)
>>
>> Next thing I see on the console is:
>>
>> free(): invalid next size (fast): 0x00000000038a3d60 ***
>>
>> I'm just not sure where to go from here.  I'm willing to do more work to
>> help figure out where the problem is, just let me know what you would
>> like to see.
>>
>> Thanks.
>>
>> --------------------------------------
>> Randy Syring
>> Intelicom
>> Direct: 502-276-0459
>> Office: 502-212-9913
>>
>> For the wages of sin is death, but the
>> free gift of God is eternal life in
>> Christ Jesus our Lord (Rom 6:23)
>>
>>
>> On 04/07/2011 03:02 PM, Randy Syring wrote:
>>> I went ahead and narrowed down my test case and submitted the report
>>> to pyodbc.  I suspect FreeTDS is also related, but I could be wrong.
>>> I just can't get any lower level than pyodbc to be able to rule that
>>> out.  A TDS dump and very simple python script to reproduce the
>>> problem are attached to the pyodbc issue:
>>>
>>> http://code.google.com/p/pyodbc/issues/detail?id=170
>>>
>>> --------------------------------------
>>> Randy Syring
>>> Intelicom
>>> Direct: 502-276-0459
>>> Office: 502-212-9913
>>>
>>> For the wages of sin is death, but the
>>> free gift of God is eternal life in
>>> Christ Jesus our Lord (Rom 6:23)
>>>
>>>
>>> On 04/07/2011 11:07 AM, Randy Syring wrote:
>>>> My platform:
>>>>
>>>> Ubuntu 10.04
>>>> Python 2.6.5
>>>> UnixODBC 2.2.11-21
>>>> pyodbc 2.1.8
>>>> SQLAlchemy 0.6.6
>>>>
>>>> I can run simple tests scripts without a problem.  However, I have
>>>> experienced some kind of crash doing some unit tests on a Python web
>>>> application.
>>>>
>>>> The error appears with both the FreeTDS version that ships with my
>>>> distro (0.82-1build6) as well as a CVS version I compiled myself.
>>>>
>>>> Here is the error in context of some SQLAlchemy debugging output:
>>>>
>>>>> 2011-04-07 10:48:42,514 INFO sqlalchemy.engine.base.Engine.0x...a350
>>>>> SELECT auth_users.id AS auth_users_id, auth_users.createdts AS
>>>>> auth_users_createdts, auth_users.updatedts AS auth_users_updatedts,
>>>>> auth_users.username AS auth_users_username,
>>>>> auth_users.username_inactive AS auth_users_username_inactive,
>>>>> auth_users.authsrc_uid AS auth_users_authsrc_uid,
>>>>> auth_users.alternate_uid AS auth_users_alternate_uid,
>>>>> auth_users.name_first AS auth_users_name_first,
>>>>> auth_users.name_middle AS auth_users_name_middle,
>>>>> auth_users.name_last AS auth_users_name_last, auth_users.phone AS
>>>>> auth_users_phone, auth_users.email AS auth_users_email,
>>>>> auth_users.inactive_flag AS auth_users_inactive_flag,
>>>>> auth_users.inactive_date AS auth_users_inactive_date,
>>>>> auth_users.password_expires AS auth_users_password_expires,
>>>>> auth_users.wfh_flag AS auth_users_wfh_flag, auth_users.wfh_ca_key AS
>>>>> auth_users_wfh_ca_key, auth_users.wfh_ext_reason AS
>>>>> auth_users_wfh_ext_reason, auth_users.wfh_ext_url AS
>>>>> auth_users_wfh_ext_url, auth_users.wfh_ext_link_text AS
>>>>> auth_users_wfh_ext_link_text, auth_users.auth_source_id AS
>>>>> auth_users_auth_source_id
>>>>> FROM auth_users
>>>>> WHERE auth_users.id = ?
>>>>> 2011-04-07 10:48:42,514 INFO sqlalchemy.engine.base.Engine.0x...a350
>>>>> (1,)
>>>>> 2011-04-07 10:48:42,527 INFO sqlalchemy.engine.base.Engine.0x...a350
>>>>> INSERT INTO corporate_sessions (createdts, updatedts, id, hits,
>>>>> relatedip, user_id) VALUES (?, ?, ?, ?, ?, ?)
>>>>> 2011-04-07 10:48:42,528 INFO sqlalchemy.engine.base.Engine.0x...a350
>>>>> (datetime.datetime(2011, 4, 7, 10, 48, 42, 527866), None,
>>>>> u'd2f8bb5f5w7njUnTDy0s', 0, '10.121.113.107', 1)
>>>>> *** glibc detected ***
>>>>> /home/rsyring/dev/customers/pai/paiauth-venv/bin/python: free():
>>>>> invalid next size (fast): 0x0000000004029b40 ***
>>>>> free(): invalid next size (fast): 0x0000000004029b40 ***
>>>> I then have to kill the process.
>>>>
>>>> The TDS dumps are here:
>>>>
>>>> http://rcs-comp.com/tmp/freetds-091CVS.log
>>>> http://rcs-comp.com/tmp/freetds-082.log
>>>>
>>>> I haven't taken the time to try and narrow down the test case because
>>>> I am hoping the dumps will be sufficient to track down the problem.
>>>> However, if needed, I can try to do that, its just going to be
>>>> difficult b/c its buried in test setup.
>>>>
>> _______________________________________________
>> FreeTDS mailing list
>> [email protected]
>> http://lists.ibiblio.org/mailman/listinfo/freetds
>>
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds