Re: Segfault in in PyString_FromStringAndSize

On Fri, Mar 06, 2009 at 10:06:04AM -0500, James Henstridge wrote:
> For these sort of bugs, I'd recommend using valgrind to track down the problem.
> 
> First you'll need a test program that triggers the bug, but it sounds
> like you've got that already.
I can reproduce the segfault with this program (which mimics the
mechanics of my program, to the simplest possible form, preserving the
threading):

#!/usr/bin/python
# -*- coding: utf-8 -*-
import sys
import psycopg2
import psycopg2.extras
import threading
from   signal      import      *
import Queue

class DB:
    def __init__(self):
        dbn = "dbname='%s' user='%s' host='%s' port='%s' password='%s' sslmode='%s'"  %  ('db','user','host',5432,'pass','require')
        try:
             self.conn = psycopg2.connect(dbn)
        except Exception, X:
             log("Can't connect to DB: %s" % X)
             sys.exit(1)
        cur = self.conn.cursor()
        sql = "SET client_encoding TO 'unicode'"
        cur.execute(sql)
        cur.close()
class F:
    def __init__(self, q):
        self.q = q
    def run(self):
        i = 1;
        comm = 1;
        while 1:
            self.q.put((comm, str(i))) # note that this is the str that will trigger the segfault
            if i > 9998:
                i=1
                if comm > 5: comm = 1
                else: comm += 1
            else: 
                i += 1
class P:
    def __init__(self, db, q):
        self.q = q
        self.db = db
    def run(self):
        while 1:
            tup = self.q.get(block=True)
            sql = "SELECT * FROM _table_ WHERE col=%s AND colB= %s"
            vals = [tup[0], tup[1]]
            print "getting cur"
            cur = self.db.conn.cursor()
            print "got cur", cur
            print "exec query", sql, vals
            cur.execute(sql, vals)
            print "got query, result:", cur.fetchall()
            print "close cur"
            cur.close()
            print "closed"



def new_thread(th_class, th_name ):
    r = threading.Thread(name=th_name, target = th_class.run)
    r.setDaemon(1)
    r.start()

event_qi = Queue.Queue()
def main(argv):
    db = DB()
    p =  P(db, event_qi)
    new_thread(p, 'pp')

    f =  F(event_qi)
    new_thread(f, 'ff')

    pause()


if __name__ == '__main__':
    main(sys.argv)


Now, running this program in gbd or in command line gets segfault after
not much of several thousands queries.

> Unfortunately, to get best results you'll probably need to compile
> your own Python interpreter with the --disable-pymalloc flag to
> configure.  This is needed because valgrind doesn't know how to track
> allocations made by Python's special purpose memory allocator.
> Disabling that allocator causes Python to do standard malloc/free
> calls, giving useful results.
Built from debian package with --without-pymalloc
 
> [I've got a patch for Python that would do this automatically at
> runtime, but it hasn't been reviewed or merged yet].
> 
> You can then try something like:
> 
>     valgrind --tool=memcheck --log-file=valgrind.log --leak-check=full
> --num-callers=20 /path/to/my/python test-script.py
Well, running inside valgrind does not segfault after 15 hours.


Any advice?
Thanks,
Fernando

_______________________________________________
Psycopg mailing list
Psycopg-IAPFreCvJWPBWskQ1e/[email protected]
http://lists.initd.org/mailman/listinfo/psycopg

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkmyipsACgkQCtDmpVXdEMYGBgCcDUiNVQAxiPvgkSWTKA4kgf2Z
184AmgMitiQl37uKG5sFj78XW9g3cJzw
=Ch7V
-----END PGP SIGNATURE-----