Re: [pysqlite] Concurrency - first steps

Hugh Gibson wrote:

> It's not without issues, though. See for example
> http://blog.ianbicking.org/gil-of-doom.html . Our main application is a
> webserver with a thread farm for responding to HTTP requests. We have
> multiple SQLite databases, one per user, so in practice there might be
> quite a lot of time when the GIL is released while a thread hits the
> database. I'm very interested in how it performs on MP machines with
> heavy loading.

this is not an uncommon pattern.  I am trying to achieve the same thing with CRM
114.  (One database per user).  However, threads are not the answer.  Threads
are just another source of problem.  Modern processes, especially if they are
all running identical applications, aren't that expensive.  What is expensive is
communicating between the processes.  look at
http://docs.python.org/library/multiprocessing.html#module-multiprocessing for a
way to get away from threads.  I really should investigate to see if there is a
version available for 2.5

> Threads are dangerous. Why should Python insulate you from that danger at
> the expense of performance? I would like to be able to make that sort of
> decision myself.

even Python threads are dangers.  Every time I've used them, they have bit me in
the butt.  This is why I'm so interested in the multiprocessing module.  the
following problem summarizes the issues far better than I could.

http://radar.oreilly.com/archives/2007/01/threads-conside.html