Re: Proposed design for importlib.resources()

On 22 November 2015 at 02:19, Brett Cannon <[email protected]> wrote:
> On Fri, 20 Nov 2015 at 20:01 Nick Coghlan <[email protected]> wrote:
>> The general usage API design looks good to me, but the current proposal
>> for retrieving the resource reader uses loader_state incorrectly - that's
>> defined in PEP 451 as an opaque object from the import machinery's point of
>> view, so there's no requirement for it to be a mapping. Instead,
>
> Dammit, it was so convenient!

We underspecified it precisely to avoid the temptation to give it
semantic significance instead of treating it as an opaque reference
allowing custom Importers to pass data to custom Loaders :)

>> "resource_reader" either needs to be a new optional attribute on the
>> module spec, or else a new optional method on the Loader API.
>
> Yep.
>
>> My preference is for the latter, as that way we'll never create resource
>> reader instances for the vast majority of modules, while with the current
>> proposal we'd create a reader instance for every module *spec* constructed,
>> even if nothing in the application uses the new resource access API.
>
> I'll have to think about that. I'm not that worried about memory pressure
> from every module having a resource reader object (it's not like people
> import literally a million modules; I have never heard more than in the
> thousands). We could introduce Loader.resources(name) or __spec__.resources
> depending on how this plays out (people are welcome to provide feedback on
> which way they prefer).

My preference is to have it as a Loader method API. I actually have a
concrete technical rationale for that, too: the loader concept
predates the module spec concept by more than a decade, so folks
customising the import system will necessarily have the ability to
cope with module loaders. By contrast, module specs can only be relied
on in Python 3 code, or in Python 2 code that's willing to introduce a
hard dependency on the use of importlib2 as the import system.

By instead having the resource reader as a (dynamically created?)
subcomponent of the loader, we get to keep the overall structure of
import system manipulation code the same - resource readers wouldn't
be a new concept existing at the same level as module loaders, they'd
be a new feature *of* module loaders.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia