Re: [pysqlite] Concurrency - first steps

"Hugh Gibson" <hgibson-CVW/MsmY5bNaa/[email protected]> Wed, 19 Nov 2008 08:12 +0000 (GMT Standard Time)
Newsgroups gmane.comp.python.db.pysqlite.user
Message-ID <[email protected]>
> > In my own programming I always put a lock around that sort of
> > access - maybe I don't need to.
> 
> In Python code you don't.  In C/C++/Java code you do.
>
> ...snip...
>
> I really like the way Python does threading under the hood.  Code 
> is not run concurrently unless you explicitly release the GIL for
> sections you have designed/verified to be safe.

Thanks for the detailed explanation.

It's not without issues, though. See for example
http://blog.ianbicking.org/gil-of-doom.html . Our main application is a
webserver with a thread farm for responding to HTTP requests. We have
multiple SQLite databases, one per user, so in practice there might be
quite a lot of time when the GIL is released while a thread hits the
database. I'm very interested in how it performs on MP machines with
heavy loading.

Jython and IronPython avoid using the GIL and so may have significant
performance advantages. For portability I would like to keep open the
possibility of moving to those platforms. Hence in all my coding I'm not
assuming that the GIL exists.

> Other platforms require all code to be 100% concurrency safe.

Threads are dangerous. Why should Python insulate you from that danger at
the expense of performance? I would like to be able to make that sort of
decision myself.

Hugh