Re: Proposal for a lazy-loading finder

--===============2935599653374219039==
Content-Type: multipart/alternative; boundary="001a113701c696db9f0553fb0791"

--001a113701c696db9f0553fb0791
Content-Type: text/plain; charset="UTF-8"

On Sun, 9 Jul 2017 at 06:59 Nick Coghlan <[email protected]> wrote:

> On 9 July 2017 at 20:03, Antoine Pitrou <[email protected]> wrote:
> > I would suggest allow people to fully customize the blacklist /
> > whitelist logic using a callable (because looking up by fullname is a
> > bit inflexible).
>
> Oh, I like that - and then we'd have "ensure_lazy" and "ensure_eager"
> as callback factories that accepted a predefined list of names.
>

If we provide an optional callable argument then I would drop the
whitelist/ensure_lazy option. It's easier to explain and the common case
will be blacklisting a module for lazy loading if you're implicitly
flipping it on. For the whitelist case we can add
importlib.util.lazy_import() and people can just be explicit (if this is
important enough to even care about).

And I purposefully didn't do a prefix match for ensure_eager as it's only
meant for specific modules that have some try/except block which fails in
the face of lazy loading. And since that should be a per-module thing
instead of a per-package thing I don't want it over-extending. Plus
providing a callback solution lets people engineer their own prefix
matching solution if that's what they need.


>
> I think this is going to be one of the key changes in going from
> "utility a project implements for itself if it needs it" to "general
> purpose language level facility" - I could easily see the simple
> predefined list approach working for a *particular* project, but I
> think once we open this capability up to all Python users (rather than
> only those that are comfortable with customising the import system at
> runtime) it will prove to be inadequate.
>

Probably, which is partially why I have put off proposing the idea of
importlib.util.activate_lazy_loading() for so long (and yes, Antoine, that
function and LazyLoadingFinder is all that's needed :) . But I met multiple
people at PyCon US this year who thanked me for the lazy loader which
suggests the note warning people about not using the lazy loader is being
ignore, so I figured it was finally time to help make sure people at least
do it correctly.

-Brett


>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [email protected]   |   Brisbane, Australia
> _______________________________________________
> Import-SIG mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/import-sig
>

--001a113701c696db9f0553fb0791
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Sun=
, 9 Jul 2017 at 06:59 Nick Coghlan &lt;<a href=3D"mailto:[email protected]=
">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quo=
te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=
>On 9 July 2017 at 20:03, Antoine Pitrou &lt;<a href=3D"mailto:solipsis@pit=
rou.net" target=3D"_blank">[email protected]</a>&gt; wrote:<br>
&gt; I would suggest allow people to fully customize the blacklist /<br>
&gt; whitelist logic using a callable (because looking up by fullname is a<=
br>
&gt; bit inflexible).<br>
<br>
Oh, I like that - and then we&#39;d have &quot;ensure_lazy&quot; and &quot;=
ensure_eager&quot;<br>
as callback factories that accepted a predefined list of names.<br></blockq=
uote><br></div><div class=3D"gmail_quote">If we provide an optional callabl=
e argument then I would drop the whitelist/ensure_lazy option. It&#39;s eas=
ier to explain and the common case will be blacklisting a module for lazy l=
oading if you&#39;re implicitly flipping it on. For the whitelist case we c=
an add importlib.util.lazy_import() and people can just be explicit (if thi=
s is important enough to even care about).<br><br></div><div class=3D"gmail=
_quote">And I purposefully didn&#39;t do a prefix match for ensure_eager as=
 it&#39;s only meant for specific modules that have some try/except block w=
hich fails in the face of lazy loading. And since that should be a per-modu=
le thing instead of a per-package thing I don&#39;t want it over-extending.=
 Plus providing a callback solution lets people engineer their own prefix m=
atching solution if that&#39;s what they need.<br></div><div class=3D"gmail=
_quote"><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:=
0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I think this is going to be one of the key changes in going from<br>
&quot;utility a project implements for itself if it needs it&quot; to &quot=
;general<br>
purpose language level facility&quot; - I could easily see the simple<br>
predefined list approach working for a *particular* project, but I<br>
think once we open this capability up to all Python users (rather than<br>
only those that are comfortable with customising the import system at<br>
runtime) it will prove to be inadequate.<br></blockquote><div><br></div><di=
v>Probably, which is partially why I have put off proposing the idea of imp=
ortlib.util.activate_lazy_loading() for so long (and yes, Antoine, that fun=
ction and LazyLoadingFinder is all that&#39;s needed :) . But I met multipl=
e people at PyCon US this year who thanked me for the lazy loader which sug=
gests the note warning people about not using the lazy loader is being igno=
re, so I figured it was finally time to help make sure people at least do i=
t correctly.<br><br></div><div>-Brett<br></div><div>=C2=A0</div><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex">
<br>
Cheers,<br>
Nick.<br>
<br>
--<br>
Nick Coghlan=C2=A0 =C2=A0|=C2=A0 =C2=A0<a href=3D"mailto:[email protected]=
" target=3D"_blank">[email protected]</a>=C2=A0 =C2=A0|=C2=A0 =C2=A0Brisba=
ne, Australia<br>
_______________________________________________<br>
Import-SIG mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Import-SIG@pytho=
n.org</a><br>
<a href=3D"https://mail.python.org/mailman/listinfo/import-sig" rel=3D"nore=
ferrer" target=3D"_blank">https://mail.python.org/mailman/listinfo/import-s=
ig</a><br>
</blockquote></div></div>

--001a113701c696db9f0553fb0791--

--===============2935599653374219039==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Import-SIG mailing list
[email protected]
https://mail.python.org/mailman/listinfo/import-sig

--===============2935599653374219039==--