Re: Imports with hooks not thread safe?

On Sun, 2002-07-14 at 07:58, Gordon McMillan wrote:
> On 13 Jul 2002 at 20:55, Anthony Tuininga wrote:
> 
> > Thanks. But wouldn't it make more sense to acquire the
> > lock around __any__ import, regardless of whether it
> > was hooked or not? 
> 
> You can't. Hooks typically *replace*
> builtin.__import__. Imports from Python code
> (where import has been hooked) don't necessarily touch
> anything in the builtin Python import machinery.

Ok but please bear with me a moment while I struggle to understand
this....

If I look at the C code for Python (Python/import.c), any code written
in C uses PyImport_Import if it wants to invoke the import hooks that
have been set up and PyImport_ImportModuleEx if it wants to "go direct"
so to speak. Technically, the lock could be acquired and released within
PyImport_Import and this would solve __ONE__ method for importing
modules. PyImport_Import calls the __import__ method of the current
globals which is how hooks work, right?

I then found __ANOTHER__ location (Python/ceval.c) where imports are
done. This one is what is actually used for import statements found
within Python itself. It calls the __import__ method of the current
globals directly without bothering to call anything in import.c. Now
perhaps these could be reconciled, but if not, the import lock could be
acquired and released here as well. Right? Or absolutely wrong???

> [...]
> 
> > I agree that the exposing of the import lock would
> > also solve the problem but that would mean that all
> > the import hooks would have to do this in order to
> > be thread safe. 
> 
> Correct. Writing a good import hook isn't easy.
> Most import hooks are specializations or extensions
> of behavior in some little area, so can get away with
> it.

So wouldn't it be better to make this simpler? Or would making it
simpler just cause more grief in the long run....? :-)

> > Or could it be placed in the core
> > imputils.py in the base class? Comments? 
> 
> iu is a replacement for imputils. imputils has a
> number of flaws - no import lock; it does not
> put None in sys.modules when the "is this a
> relative import?" test fails. Can't recall what else,
> but that's why I wrote iu.

So wouldn't it be better to fix imputils.py to do this right? Or is it
that much of a bother to get changes to core Python?

> 
> -- Gordon
> http://www.mcmillan-inc.com/
> 
> 
> 
> _______________________________________________
> Import-sig mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/import-sig
-- 
Anthony Tuininga
[email protected]
 
Computronix
Distinctive Software. Real People.
Suite 200, 10216 - 124 Street NW
Edmonton, AB, Canada  T5N 4A3
Phone:	(780) 454-3700
Fax:	(780) 454-3838
http://www.computronix.com