Re: PEP 489: Redesigning extension module loading

On Sun, Apr 26, 2015 at 10:31 AM, Stefan Behnel <[email protected]> wrote:
> Petr Viktorin schrieb am 17.04.2015 um 12:33:
>> On 04/17/2015 08:51 AM, Stefan Behnel wrote:
>>> Petr Viktorin schrieb am 16.04.2015 um 13:05:
>>>> A wart I added is "singleton modules", necessary for
>>>> "PyState_FindModule"-like functionality. I wouldn't mind not including
>>>> this, but it would mean the new API can't replace all use cases of the
>>>> old PyInit_.
>>>>
>>>> Singleton Modules
>>>> -----------------
>>>>
>>>> Modules defined by PyModuleDef may be registered with PyState_AddModule,
>>>> and later retrieved with PyState_FindModule.
>>>>
>>>> Under the new API, there is no one-to-one mapping between PyModuleSpec
>>>> and the module created from it.
>>>> In particular, multiple modules may be loaded from the same description.
>>>
>>> Is that because a single shared library (which is what the module spec
>>> refers to, right?) can contain multiple modules? Or are you referring to
>>> something else here?
>>
>> By using Loader.create_module/Loader.exec_module directly, you can load an
>> extension module without adding it to sys.modules. You can do this as many
>> times as you like, and you always get a new, independent module object.
>
> Should we even allow that for extension modules, as long as there is no
> reloading support?

That's a good question.
Without reload/unload support, It isn't really necessary. But, in most
cases avoiding PyState_FindModule is not that hard to do (especially
once class methods are fixed, as mentioned in the other subthread).

> I guess subinterpreters would also have a say in this, right?

I also think independent modules are easier to reason about than
subinterpreters; and if you support independent modules correctly
(without PyState_FindModule) then there's a rather big chance you also
got subinterpreter support right.
Also it is easier to write tests with independent modules than with
subinterpreters.

So while PyState_FindModule works, I'd rather promote passing explicit
references. (If that doesn't work somewhere, singleton modules can be
added again, but that might be a special use case that'll need a
slightly different solution.)