Re: Loading Resources From a Python Module/Package

> On Jan 31, 2015, at 10:54 AM, Paul Moore <[email protected]> wrote:
> 
> On 31 January 2015 at 15:47, Donald Stufft <[email protected]> wrote:
>>> It's certainly possible to add a new API that loads resources based on
>>> a relative name, but you'd have to specify relative to *what*.
>>> get_data explicitly ducks out of making that decision.
>> 
>> data = __loader__.get_bytes(__name__, “logo.gif”)
> 
> Quite possibly. It needs a bit of fleshing out to make sure it doesn't
> prohibit sharing of loaders, etc, in the way Brett mentions. Also, the
> fact that it needs __name__ in there feels wrong - a bit like the old
> version of super() needing to be told which class it was being called
> from. But in principle I don't object to finding a suitable form of
> this.

To be clear, I think using __name__ is massively better than using __file__,
for one even though PEP 302 states that __file__ must be set, it actually
doesn’t have to be set and PEP 420 doesn’t set it. Even if it did set it
that pattern is only actually really usable for non namespace packages (of
any type).

The namespace package way of doing that is basically:

for path in __path__:
    try:
        data = __loader__.get_data(os.path.join(path, “logo.gif”))
    except FileNotFoundError:
        pass
    else:
	break
else:
    raise Exception(“Cannot Find the file ‘logo.gif’”)


Either way if a Loader isn’t specific to a particular importable name and can
be re-used between them then you need a way to specify what module it’s relative
to and it seems to me the *obvious* way to load a resource that is relative to
a module is to tell Python you want to load a particular resource from a particular
module, not to construct some (pseudo) file path that says all that information
as well but requires you to know if the thing you’re importing is a Python
module, a python package, or a namespace package.

In order to make a function like pkgutil.get_data that actually works in all
situations that you’d have to do something like:

def get_data(package, resource):
    mod = importlib.import_module(package)
    if hasattr(mod, “__path__”):
        for path in __path__:
            try:
                return mod.__loader__.get_data(os.path.join(path, resource))
            except FileNotFoundError:
                pass
    if hasattr(mod, "__file__"):
        d = os.path.dirname(__file__)
        try:
            return mod.__loader__.get_data(os.path.join(d, resource))
        except FileNotFoundError:
            pass

This is compared to the situation where the Loaders encapsulate that logic
for you:

def get_data(package, resource):
    mod = importlib.import_module(package)
    try:
        mod.__loader__.get_bytes(package, resource)
    except FileNotFoundError:
        pass

Obviously the logic in the first function still exists, it’s just moved away
from the caller needing to handle it and instead the Loader handles it, just
like the loader abstracts away the __file__ location for importing a particular
module.

Although looking closer at the Loader().exec_module implementation, It appears
that it expects something other than a string to be passed to it. So if it makes
sense possibly Loader().get_bytes() etc should also expect something other than
a string to identify the module as well (whatever it actually wants, I can’t tell).
Then the utility functions in pkgutil or importlib.resources or whatever will do
the logic to translate from a string to whatever the Loader itself wants.


> 
> And I like the name get_bytes - much more explicit in these Python 3
> days of explicit str/bytes distinctions :-)
> Paul

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

_______________________________________________
Import-SIG mailing list
[email protected]
https://mail.python.org/mailman/listinfo/import-sig