Re: PEP 489: Redesigning extension module loading; version 4

On 14 May 2015 at 18:10, Petr Viktorin <[email protected]> wrote:
> On Wed, May 13, 2015 at 6:04 PM, Nick Coghlan <[email protected]> wrote:
>> On 14 May 2015 at 00:31, Petr Viktorin <[email protected]> wrote:
>>> Or, I could keep the "PyInit_*" hook name, and allow it to return
>>> PyModuleDef instead of a module. This is obviously a hack, and would
>>> force me to get back down to the drawing board, but considering the
>>> options it seems best to explore this option.
>>> (PyInit_* and PyModuleExport_* signatures are technically compatible,
>>> since a PyModuleDef is a PyObject)
>>>
>>> I'd welcome your thoughts.
>>
>> Would it be feasible to go with a model where _PyImport_inittab
>> continues to be based on the legacy extension module initialisation
>> system for the time being? That would mean implementing PyInit_* would
>> remain required rather than optional for 3.5, but lots of folks are
>> going to want to provide it anyway for compatibility with 3.4 and
>> earlier.
>
> That doesn't really solve the problem, just delays it until we decide
> that PyInit_* is really optional.

Yeah, I was seeing if you thought a "buy more time to think about it
further" approach might be viable here. I think you're right that we
need a better answer up front, though.

> It would mean you couldn't take advantage of the improvements in PEP
> 489 (create/exec split and ModuleSpec). You'd just write more
> boilerplate for no benefit (except small stuff like non-ASCII module
> names).
>
> What might be worse, it would mean that modules would have different
> behavior depending on whether they're frozen or not, which would
> probably result in subtle bugs you'd only find when creating frozen
> binaries.

Looking at https://hg.python.org/cpython/file/default/Tools/freeze/makeconfig.py,
I'm thinking your "out-of-band" option may be a reasonable way to go,
with a corresponding tweak to the semantics of
https://docs.python.org/3/c-api/import.html#c._inittab to permit
(initfunc) to be a pointer to a PyInit_* function OR to a
PyModuleExport_* function.

We'd then have to determine which was which at runtime when processing
the inittab internally, by checking whether or not the result of the
call was a PyModuleDef or not.

For the inittab generation side, freeze would need to be updated to:

* allow builtin modules to be specifically nominated as "initialised
modules" or "defined modules"
* allow the default handling of builtin modules not nominated as one
or the other to be configured
* for backwards compatibility, builtin modules would be treated as
initialised modules by default

If you had a new module that was export only, you'd get a link time
error looking for the init function that didn't exist if you didn't
explicitly flag it as a "defined module". Similarly, if you switched
the default to be defined modules, you'd get a link time error for a
legacy module that didn't support the new API.

Does that approach sound plausible to you?

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia