Re: making it feasible to rely on loaders for reading intra-package data files

On Sat, Feb 1, 2014 at 2:04 PM, Paul Moore <[email protected]> wrote:

> On 1 February 2014 18:44, Brett Cannon <[email protected]> wrote:
> > The second issue is whether get_data/set_data are enough or if something
> > else is needed, e.g. a listdir-like method. Since this is meant for
> handling
> > intra-package data my assumption is that it isn't really necessary as
> > chances are you know what files you included in your distribution (or at
> > least what the possible names are). I know some have asked for a
> > listdir-like API to help discover what modules are available so as to
> > provide a plugin API, but I view that as a separate thing and potentially
> > more appropriate on finders. Remember, the smaller the API service for
> the
> > common case the better for the stdlib.
>
> One immediate use case for a listdir-type function is virtualenv.
> Admittedly, listdir is only half of the problem (and the other half -
> the receiving code needs a real file, not a resource - is harder to
> adderss) but it is an example of where the API might be helpful.
>
> Basically, virtualenv uses a virtualenv_support directory (package -
> in that it has an __init__.py) to hold the wheels for pip and
> setuptools that are to be loaded into the virtualenv being created.
> While in theory we know the names of those wheels, the problem is that
> wheel names encode the version of the package, so rather than have to
> update the code every time we look for setuptools*.whl and pip*.whl.
> Also, we support users replacing the supplied wheels with newer
> versions, so the actual filenames aren't in our control anyway.
>
> As I say, to actually use the wheels we need to have them in the
> filesystem, so there are other issues that would prevent us from
> removing the filesystem assumption in the short term. But we couldn't
> even start without a listdir API.
>
> BTW, an unrelated issue is that if we did go down this route with
> virtualenv, we'd be looking at having a resource that is the content
> of a zipfile that we'd want to put on sys.path. There's no support in
> Python for putting in-memory zipfiles on sys.path. We could, and
> probably would, dump the data to a temporary file in the first
> instance and put that on sys.path, but in the light of this thread, is
> putting in-memory zipfiles onto sys.path something that we should be
> supporting?
>

Not quite sure what you are suggesting as an in-memory zipfile vs. one that
isn't. Any zipfile on sys.path has to be in-memory to read from to do a
load so I don't know where you are drawing the distinction.

_______________________________________________
Import-SIG mailing list
[email protected]
https://mail.python.org/mailman/listinfo/import-sig