Re: [pysqlite] Concurrency - first steps

> > Our main application is a
> > webserver with a thread farm for responding to HTTP requests. 
> 
> Do you have one thread per connection, or worker thread pool?

Worker thread pool. Main thread handles select() call, and then creates
appropriate instances of HTTPConnection and HTTPRequest objects. A
HTTPConnection object can have multiple requests as we implement HTTP
1.1.

Requests are handed to the worker thread farm. There are also some
subsidiary threads doing housekeeping tasks like checking for failed
sessions.

> > I'm very interested in how it performs on MP machines with
> > heavy loading.
> 
> That is exactly what you measure before the premature optimizations 
> :-)

Point taken... 

> > Jython and IronPython avoid using the GIL and so may have 
> > significant performance advantages.
> 
> Actually they have immediate performance disadvantages.  All Python 
> data structures have to have mutexes so that two threads don't trash 
> them. Every single access has to acquire the mutex, do the access
> and then release the mutex.  With the GIL the code just does the
> access.

OK, didn't know that. Also take on board your later comments re Java etc.

> Multiple processes are far better since each processor can go full
> speed without having to worry about the others.  
>
> ... snip ...
> 
> There is even a Python 2.6/3 module to make it easy:
> http://docs.python.org/whatsnew/2.6.html#pep-371-the-multiprocessing
> -package

Thanks. For historical reasons (wxMozilla editor) we're stuck with an
older version of Python. But this would certainly be a good reason to
upgrade to a later version - if we have a problem (I want to avoid
premature optimisation!).

Hugh