Re: [pysqlite] Erratic behavior under high loads, with APSW code sample

 -------------- Original message ----------------------
From: "Frank McIngvale" <[email protected]>
> On Mon, Jun 30, 2008 at 11:25 AM, Glenn Maynard <[email protected]> wrote:
> 
> > On Mon, Jun 30, 2008 at 06:09:11AM -0500, Frank McIngvale wrote:
> > > Hi, I'm having a strange problem with APSW & pysqlite that shows up under
> > > higher loads. I've attached two files -- running them like this shows the
> > > problem (this occurs under both APIs, I just wrote this sample for APSW):
> > >
> > > $ rm -f test.db; python raw_apsw.py & python raw_apsw.py & python
> > > raw_apsw.py
> > > $ python r.py
> > >
> > > This should show "ROWS=90" at the end. Under Windows (both XP and Vista)
> > > this is rock-solid. Under Linux however, the results are erratic, and
> > almost
> > > always significantly less than 90. I've tried this under both Ubuntu 8.04
> > > (linux-2.6.24, apsw 3.3.13, sqlite 3.4.2) and RedHat ES 4 (linux 2-6.9,
> > apsw
> > > 3.5.9-r1, sqlite 3.5.9). As I understand it, SQLite's auto-transaction
> > > feature should keep the INSERTs isolated, but I've also tried this same
> > > sample using explicit BEGIN EXCLUSIVE/COMMIT/ROLLBACK, with the same
> > > results.
> >
> > Without adding a conn.commit(), this doesn't add any rows for me at all
> > (with sqlite3).  I'm not sure why anything would ever be written without
> > not committing.
> 
> 
> This is vs. APSW, though. It doesn't have a .commit() and it defaults (my
> understanding) to the pysqlite equivalent of 'isolation_level=None', so
> SQLite is doing an auto-transaction for each INSERT (in my example). I
> definitely am getting data written, just not the whole amount (under POSIX;
> Windows is fine).
> 
> I can post a pysqlite equivalent sample if that would help. I suspect the
> problem lies in SQLite itself, but I wanted to see first if anyone here
> could run & replicate the issue on a non-Windows box.
> 
> frank
> 

If it helps from:
http://www.sqlite.org/cvstrac/wiki?p=MultiThreading

SQLite multithreading settings 
The setting named THREADSAFE turns multithreading on or off. It's turned on by default in the precompiled Windows binaries and it's off by default in the precompiled Linux binaries. Under Linux, Mac OS X and other Unix systems, you'll have to set it manually. If you're using Mac OS X's Project Builder, you can easily turn on multithreading by adding -DTHREADSAFE=1 to the Other C Compiler Flags field, in the following panel: 
Project:Edit active target '<your project>':Settings:GCC Compiler Settings 
Some messages in the SQLite group list refer to the following functions: sqliteOsEnterMutex() and sqliteOsLeaveMutex(). These functions set and clear the mutex lock, which is needed to guarantee a thread-safe environment. Under Mac OS X and Windows, the sqliteOsEnterMutex() and sqliteOsLeaveMutex() functions are already implemented in os.c. 

Seems threading is not turned on by default in POSIX systems. 


--
Adrian Klaver
[email protected]

_______________________________________________
list-pysqlite mailing list
list-pysqlite-FR6EJeJVuqdwc357pe9rcyQmJico6nz3epZhswDD4dQ@public.gmane.org
http://itsystementwicklung.de/cgi-bin/mailman/listinfo/list-pysqlite