Re: Proposed design for importlib.resources()

--===============5792946197629626445==
Content-Type: multipart/alternative; boundary=001a114db34a79718105250f5db1

--001a114db34a79718105250f5db1
Content-Type: text/plain; charset=UTF-8

On Fri, 20 Nov 2015 at 20:01 Nick Coghlan <[email protected]> wrote:

> On 21 November 2015 at 07:23, Brett Cannon <[email protected]> wrote:
>
>> I have created a Jupyter Notebook to explain my thinking on what
>> importlib.resources() should be (at least initially). You can view the
>> notebook at
>> http://nbviewer.jupyter.org/gist/brettcannon/9c4681a77a7fa09c5347 or
>> download it and play with the code live in your own copy (you can download
>> Anaconda 2.4 if you don't have Jupyter already set up under Python 3.5:
>> https://www.continuum.io/downloads; I have filed
>> https://github.com/binder-project/binder/issues/38 to try and get
>> mybinder.org updated to Python 3.5 so that can be used instead).
>>
>> The notebook is a bit long and is much better formatted elsewhere, so I'm
>> not going to inline it here. If you want to comment on the notebook just
>> copy and paste the relevant part into your reply.
>>
>
> The general usage API design looks good to me, but the current proposal
> for retrieving the resource reader uses loader_state incorrectly - that's
> defined in PEP 451 as an opaque object from the import machinery's point of
> view, so there's no requirement for it to be a mapping. Instead,
>

Dammit, it was so convenient!


> "resource_reader" either needs to be a new optional attribute on the
> module spec, or else a new optional method on the Loader API.
>

Yep.


>
> My preference is for the latter, as that way we'll never create resource
> reader instances for the vast majority of modules, while with the current
> proposal we'd create a reader instance for every module *spec* constructed,
> even if nothing in the application uses the new resource access API.
>

I'll have to think about that. I'm not that worried about memory pressure
from every module having a resource reader object (it's not like people
import literally a million modules; I have never heard more than in the
thousands). We could introduce Loader.resources(name) or __spec__.resources
depending on how this plays out (people are welcome to provide feedback on
which way they prefer).


>
> Some other smaller notes:
>
> * The notebook reports the result of your straw poll incorrectly - you say
> "approach 2" won out, but "approach 1" (the object oriented one) did (by a
> 4:1 margin).
>

Typo fixed in my copy.


> * In relation to sharing files, there are actually options we can pass to
> CreateFile to make it possible to open a temporary file by name even while
> we keep the original handle open:
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx
> * There's also an open issue discussing the significant limitations of
> tempfile.NamedTemporaryFile on Windows: http://bugs.python.org/issue14243
>
> I don't think either of those notes about shared file access on Windows
> affect your proposed solution, I just think they're worth referencing.
>

Yes, I'm ignoring everything you just said. :) If we were supporting the
return of objects then I might care, but since I'm trying to keep the API
surface small to start and thus not doing file objects this doesn't really
play into this. If we add an open() method then these issues will be
something we need to potentially care about.

-Brett

--001a114db34a79718105250f5db1
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Fri=
, 20 Nov 2015 at 20:01 Nick Coghlan &lt;<a href=3D"mailto:[email protected]=
m">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex=
"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On=
 21 November 2015 at 07:23, Brett Cannon <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:[email protected]" target=3D"_blank">[email protected]</a>&gt;</span> w=
rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"=
>I have created a Jupyter Notebook to explain my thinking on what importlib=
.resources() should be (at least initially). You can view the notebook at=
=C2=A0<a href=3D"http://nbviewer.jupyter.org/gist/brettcannon/9c4681a77a7fa=
09c5347" target=3D"_blank">http://nbviewer.jupyter.org/gist/brettcannon/9c4=
681a77a7fa09c5347</a>=C2=A0or download it and play with the code live in yo=
ur own copy (you can download Anaconda 2.4 if you don&#39;t have Jupyter al=
ready set up under Python 3.5:=C2=A0<a href=3D"https://www.continuum.io/dow=
nloads" target=3D"_blank">https://www.continuum.io/downloads</a>; I have fi=
led=C2=A0<a href=3D"https://github.com/binder-project/binder/issues/38" tar=
get=3D"_blank">https://github.com/binder-project/binder/issues/38</a>=C2=A0=
to try and get <a href=3D"http://mybinder.org" target=3D"_blank">mybinder.o=
rg</a> updated to Python 3.5 so that can be used instead).<div><br></div><d=
iv>The notebook is a bit long and is much better formatted elsewhere, so I&=
#39;m not going to inline it here. If you want to comment on the notebook j=
ust copy and paste the relevant part into your reply.</div></div></blockquo=
te><div><br></div></div></div></div><div dir=3D"ltr"><div class=3D"gmail_ex=
tra"><div class=3D"gmail_quote"><div>The general usage API design looks goo=
d to me, but the current proposal for retrieving the resource reader uses l=
oader_state incorrectly - that&#39;s defined in PEP 451 as an opaque=20
object from the import machinery&#39;s point of view, so there&#39;s no=20
requirement for it to be a mapping. Instead,</div></div></div></div></block=
quote><div><br></div><div>Dammit, it was so convenient!</div><div>=C2=A0</d=
iv><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left=
:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_extr=
a"><div class=3D"gmail_quote"><div> &quot;resource_reader&quot; either=20
needs to be a new optional attribute on the module spec, or else a new=20
optional method on the Loader API.<br></div></div></div></div></blockquote>=
<div><br></div><div>Yep.</div><div>=C2=A0</div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e=
x"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><=
div><br>My preference is for the latter, as=20
that way we&#39;ll never create resource reader instances for the vast=20
majority of modules, while with the current proposal we&#39;d create a=20
reader instance for every module *spec* constructed, even if nothing in the=
 application=20
uses the new resource access API.<br></div></div></div></div></blockquote><=
div><br></div><div>I&#39;ll have to think about that. I&#39;m not that worr=
ied about memory pressure from every module having a resource reader object=
 (it&#39;s not like people import literally a million modules; I have never=
 heard more than in the thousands). We could introduce Loader.resources(nam=
e) or __spec__.resources depending on how this plays out (people are welcom=
e to provide feedback on which way they prefer).</div><div>=C2=A0</div><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_extra"><div=
 class=3D"gmail_quote"><div><br></div><div>Some other smaller notes:<br></d=
iv><div><br></div><div class=3D"gmail_extra">* The notebook reports the res=
ult of your straw poll incorrectly - you say &quot;approach 2&quot; won out=
, but &quot;approach 1&quot; (the object oriented one) did (by a 4:1 margin=
).<br></div></div></div></div></blockquote><div><br></div><div>Typo fixed i=
n my copy.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D=
"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D=
"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div class=3D"g=
mail_extra">* In relation to sharing files, there are actually options we c=
an pass to CreateFile to make it possible to open a temporary file by name =
even while we keep the original handle open: <a href=3D"https://msdn.micros=
oft.com/en-us/library/windows/desktop/aa363858%28v=3Dvs.85%29.aspx" target=
=3D"_blank">https://msdn.microsoft.com/en-us/library/windows/desktop/aa3638=
58%28v=3Dvs.85%29.aspx</a><br>* There&#39;s also an open issue discussing t=
he significant limitations of tempfile.NamedTemporaryFile on Windows: <a hr=
ef=3D"http://bugs.python.org/issue14243" target=3D"_blank">http://bugs.pyth=
on.org/issue14243</a><br><br></div><div class=3D"gmail_extra">I don&#39;t t=
hink either of those notes about shared file access on Windows affect your =
proposed solution, I just think they&#39;re worth referencing.</div></div><=
/div></div></blockquote><div><br></div><div>Yes, I&#39;m ignoring everythin=
g you just said. :) If we were supporting the return of objects then I migh=
t care, but since I&#39;m trying to keep the API surface small to start and=
 thus not doing file objects this doesn&#39;t really play into this. If we =
add an open() method then these issues will be something we need to potenti=
ally care about.</div><div><br></div><div>-Brett=C2=A0</div></div></div>

--001a114db34a79718105250f5db1--

--===============5792946197629626445==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Import-SIG mailing list
[email protected]
https://mail.python.org/mailman/listinfo/import-sig

--===============5792946197629626445==--