Re: Loading Resources From a Python Module/Package

> On Jan 31, 2015, at 12:44 PM, Barry Warsaw <[email protected]> wrote:
> 
> On Jan 30, 2015, at 07:52 PM, Donald Stufft wrote:
> 
>>> On Jan 30, 2015, at 7:18 PM, Paul Moore <[email protected]> wrote:
>>> Related question - how would the temp files be cleaned up? At exit?
>> 
>> My patch registers an atexit handler that cleans up the temporary files yea.
> 
> Why not implement it as a context manager?
> 
> I'm not a big fan of overloading the atexit handler because there are
> situations where it might not get called (e.g. the program crashes or is kill
> -9'd), but a context manager allows the resource to be cleaned up asap.
> 
> Reviewing my own uses of pkg_resources.resource_filename() I think it would
> work just fine because I rarely need the path much longer than the immediate
> operation.  If I did need to cache it more permanently, I could easily do:
> 
>    with resource_filename('my.package.path', 'foo.dat') as path:
>        shutil.copy(path, some_more_permanent_location)
> 
> Easy peasy.


The reasons for not wanting to use a context manager are sort of intertwined
with each other.

The competitor to this function is something like:

    import os.path
    import time

    LOGO_PATH = os.path.join(os.path.dirname(__file__), "logo.gif")

    def print_logo_path():
        print(LOGO_PATH)


    while True:
        print_logo_path()
        time.sleep(1)

So when looking at an alternative that we want people to use we have to
consider the cost of porting to that code from the old way. Using an atexit
handler means that the above code can be switched to the new mechanism
just by chaning a single line:

    LOGO_PATH = importlib.resources.get_filename(__name__, "logo.gif")

Using a context manager would require something like:

    LOGO_MAKER = lambda: importlib.resources.get_filename(__name__, "logo.gif")

    def print_logo_path():
        with LOGO_MAKER as filename:
            print(filename)

Or:

    _LOGO_TMP = importlib.resources.get_filename(__name__, "logo.gif")
    atexit.register(_LOGO_TMP.cleanup)
    LOGO_PATH = _LOGO_TMP.name

It makes it more akward to use anytime you need to use the file in multiple
locations or multiple times and since each context manager instance (in the
worst case) is going to need to get bytes, create a temp file, and write bytes
for each use of the context manager.

The other thing is that for the "common" case, where the resource is available
on the file system already because we're just using a FileLoader, there is no
need for an atexit handler or a temporary file at all. The context manager
would only really exist for the uncommon case where we need to write the data
to a temporary file. Using the atexit handler allows us to provide the best
API for the common case, without too much problem for the uncommon case.

Yes it does mean that in certain cases the temporary files may be left behind,
particularly with kill -9 or segfaults or what have you. However that case
already exists, the only thing the context manager does is narrow the window
of case where a kill -9 or a segfault can leave temporary files behind.


---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA