Re: all python threads hung

On Fri, 07 Sep 2007 13:30:02 +0000, [email protected] wrote:
>> Are you using the latest version of pysqlite? 2.3.5 has improved
> concurrency a
>> little.
> 
> Yes.  I just upgraded to 2.3.5 in the hope that would help improve the
> situation!  I didn't see any performance improvement, but my system is
> limited by flash memory access.
> 
>> In the past I've experienced that schema changes (any, even CREATE
> TEMPORARY
>> VIEW) can cause problems for other database connections. They'll then
> get schema
>> errors and will need to be recompiled (pysqlite does that
> transparently).
> 
> Yes.  I ran into the long ago and removed all uses of views to avoid it.

Perhaps you overreacted a bit? As long as you're not creating/dropping schema objects all the time, it's not a problem. I. e. normal views that are part of the schema are ok.

> [...] 
> I wonder if there is anything special about creating a database
> connection?  The readers all access the database  through an abstraction
> class.  Since I believe you can not create a connection in one thread and
> use it in another I open and close the connection each time a request is
> made. 

SQLite might need a lock when creating connections, but I'm not sure. I'd recommend opening connections and keeping them open.

> (I could cache the connections for each thread, but I'm not sure how
> to determine what execution thread I'm in.)

Use some variation of threading.local. That's how all the other database wrappers like SQLAlchemy are doing it as well.

-- Gerhard