[f2py] threading with f2py

As you may know python threading, unless their is a new module for it,  
suffers from the curse of the global lock.

to remind you, it means that python threading never is able take  
advantage of having more than one CPU because every time it goes to  
access any variable, it locks all variables, blocking all other  
threads for accessing any variable.  Thus it can only timeslice a  
single process not have concurrent processes.  i.e. threads truly  
stink in python in terms of processor utilization, but still have  
their uses for responding to events.

Now I had read that the exception to this is that any calls to c-code  
or fortran, such as happen during I/O release the global lock and thus  
are not blocking while in the binary.

I must have misunderstood that.  Or else I'm doing something wrong.

what I did was I created some fortran code and wrapped if with f2py.    
I then call this code from inside multiple python threads.

now what I expected to happen was to see my processor utilization jump  
up from a mere 100% to 800% since I have 8 cpus and I have 8 threads.

instead my processor utilization is still 100% not 800%.

-----

regression:

at first I was calling my code like this:

from threading import Thread

class testit(Thread):

   def __init__(self, r,x):
         Thread.__init__(self)
	self.x = x
	self.r = r

   def run (self):
     print "starting"	
     self.r = mycode( self.x)

# instantiate 8 threads

x = arange(3)
r = arange(3)
for r in range(8):
    current = testit(r,x)
    tlist.append(current)

# start the theads
for t in tlist:
    t.start()



where x and r were arrays I passed in during the __init__ that are  
then used in the mycode() call.

what this did was once I started one thread, none of the others could  
be started till the first one finished.

the apparent reason for this was that the mycode() call was passing in  
and out arrays that were not local to the instance, that these were  
getting locked and blocking the parent thread from starting any more  
threads.

Okay so I "fixed" this by making local copies of the arrays so there  
were no non-local variables in the run() method.


from threading import Thread

class testit(Thread):

   def __init__(self, r,x):
         Thread.__init__(self)
	self.x = x.copy()
	self.r = r.copy()

def run (self):
     print "thread is running"
     self.r = mycode( self.x)

note the added .copy() in the init(), so now run() only  has local  
instance variables.

when I run this, indeed all the run() commands execute and announce  
they are running.  time passes and then a long time later they all  
finish without errors.

but the CPU never gets up to more than 100%

The only thing I can think of is that either

1)  I don't understand threading in python

2) that when I linked in mycode() using  f2py  that this put in some  
layer that has some python variable that is shared by all calls to  
mycode().  as a result, python is blocking in the wrapper before it  
gets into the fortran binary.

any suggestions  (and don't say  "hey if you want real threading try  
perl"  )

maybe there's some new threading library in python besides Thread that  
does thread right or some trick i can do with f2py that does not share  
any variables?