Re: Proposal for a lazy-loading finder
Brett Cannon <[email protected]> Tue, 11 Jul 2017 16:50:50 +0000
| Newsgroups | gmane.comp.python.import |
|---|---|
| Message-ID | <CAP1=2W5nVRv+xd07wWg=-Cw4+vJmNFQCJgXXJE=9PtVrGEZHiA@mail.gmail.com> |
--===============9099135998355608205== Content-Type: multipart/alternative; boundary="94eb2c1970aa1fd30a05540d8130" --94eb2c1970aa1fd30a05540d8130 Content-Type: text/plain; charset="UTF-8" On Mon, 10 Jul 2017 at 18:55 Nick Coghlan <[email protected]> wrote: > On 11 July 2017 at 04:48, Brett Cannon <[email protected]> wrote: > > > > > > On Sun, 9 Jul 2017 at 06:59 Nick Coghlan <[email protected]> wrote: > >> > >> On 9 July 2017 at 20:03, Antoine Pitrou <[email protected]> wrote: > >> > I would suggest allow people to fully customize the blacklist / > >> > whitelist logic using a callable (because looking up by fullname is a > >> > bit inflexible). > >> > >> Oh, I like that - and then we'd have "ensure_lazy" and "ensure_eager" > >> as callback factories that accepted a predefined list of names. > > > > > > If we provide an optional callable argument then I would drop the > > whitelist/ensure_lazy option. It's easier to explain and the common case > > will be blacklisting a module for lazy loading if you're implicitly > flipping > > it on. For the whitelist case we can add importlib.util.lazy_import() and > > people can just be explicit (if this is important enough to even care > > about). > > Sorry, I was overly terse. By callback factories, I meant something like: > > def ensure_eager_exact(names): > """Disable lazy loading for named modules""" > names = set(names) > def lazy_load_filter(fullname): > return fullname not in names > return lazy_load_filter > > def ensure_lazy_exact(names): > """Only enable lazy loading for named modules""" > names = set(names) > def lazy_load_filter(fullname): > return fullname in names > return lazy_load_filter > > def ensure_eager_by_prefix(names): > """Disable lazy loading for named modules and their submodules""" > names = set(names) > def lazy_load_filter(fullname): > parts = fullname.split(".") > prefixes = (".".join(parts[:i]) for i in range(len(parts))) > return all(prefix not in names for prefix in prefixes) > return lazy_load_filter > > def ensure_lazy_by_prefix(names): > """Only enable lazy loading for named modules and their > submodules""" > names = set(names) > def lazy_load_filter(fullname): > parts = fullname.split(".") > prefixes = (".".join(parts[:i]) for i in range(len(parts))) > return any(prefix in names for prefix in prefixes) > return lazy_load_filter > > Those could even just be recipes in the documentation rather than > actual standard library functions. > Possibly. The first two are rather simple and should be obvious for anyone wanting to use lazy loading (there will continue to be a warning in the docs that you should only use lazy loading if you know what you're doing). > > > And I purposefully didn't do a prefix match for ensure_eager as it's only > > meant for specific modules that have some try/except block which fails in > > the face of lazy loading. And since that should be a per-module thing > > instead of a per-package thing I don't want it over-extending. Plus > > providing a callback solution lets people engineer their own prefix > matching > > solution if that's what they need. > > Yep, that's why I like Antoine's callback suggestion. > So are you suggesting dropping even the ensure_eager convenience argument? --94eb2c1970aa1fd30a05540d8130 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Mon= , 10 Jul 2017 at 18:55 Nick Coghlan <<a href=3D"mailto:[email protected]= m">[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_qu= ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex= ">On 11 July 2017 at 04:48, Brett Cannon <<a href=3D"mailto:brett@python= .org" target=3D"_blank">[email protected]</a>> wrote:<br> ><br> ><br> > On Sun, 9 Jul 2017 at 06:59 Nick Coghlan <<a href=3D"mailto:ncoghla= [email protected]" target=3D"_blank">[email protected]</a>> wrote:<br> >><br> >> On 9 July 2017 at 20:03, Antoine Pitrou <<a href=3D"mailto:soli= [email protected]" target=3D"_blank">[email protected]</a>> wrote:<br> >> > I would suggest allow people to fully customize the blacklist= /<br> >> > whitelist logic using a callable (because looking up by fulln= ame is a<br> >> > bit inflexible).<br> >><br> >> Oh, I like that - and then we'd have "ensure_lazy" a= nd "ensure_eager"<br> >> as callback factories that accepted a predefined list of names.<br= > ><br> ><br> > If we provide an optional callable argument then I would drop the<br> > whitelist/ensure_lazy option. It's easier to explain and the commo= n case<br> > will be blacklisting a module for lazy loading if you're implicitl= y flipping<br> > it on. For the whitelist case we can add importlib.util.lazy_import() = and<br> > people can just be explicit (if this is important enough to even care<= br> > about).<br> <br> Sorry, I was overly terse. By callback factories, I meant something like:<b= r> <br> =C2=A0 =C2=A0 def ensure_eager_exact(names):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 """Disable lazy loading for name= d modules"""<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 names =3D set(names)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 def lazy_load_filter(fullname):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return fullname not in names<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 return lazy_load_filter<br> <br> =C2=A0 =C2=A0 def ensure_lazy_exact(names):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 """Only enable lazy loading for = named modules"""<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 names =3D set(names)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 def lazy_load_filter(fullname):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return fullname in names<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 return lazy_load_filter<br> <br> =C2=A0 =C2=A0 def ensure_eager_by_prefix(names):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 """Disable lazy loading for name= d modules and their submodules"""<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 names =3D set(names)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 def lazy_load_filter(fullname):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 parts =3D fullname.split(".&= quot;)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 prefixes =3D (".".join(= parts[:i]) for i in range(len(parts)))<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return all(prefix not in names fo= r prefix in prefixes)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 return lazy_load_filter<br> <br> =C2=A0 =C2=A0 def ensure_lazy_by_prefix(names):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 """Only enable lazy loading for = named modules and their submodules"""<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 names =3D set(names)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 def lazy_load_filter(fullname):<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 parts =3D fullname.split(".&= quot;)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 prefixes =3D (".".join(= parts[:i]) for i in range(len(parts)))<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return any(prefix in names for pr= efix in prefixes)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 return lazy_load_filter<br> <br> Those could even just be recipes in the documentation rather than<br> actual standard library functions.<br></blockquote><div><br></div><div>Poss= ibly. The first two are rather simple and should be obvious for anyone want= ing to use lazy loading (there will continue to be a warning in the docs th= at you should only use lazy loading if you know what you're doing).<br>= </div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 = 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> > And I purposefully didn't do a prefix match for ensure_eager as it= 's only<br> > meant for specific modules that have some try/except block which fails= in<br> > the face of lazy loading. And since that should be a per-module thing<= br> > instead of a per-package thing I don't want it over-extending. Plu= s<br> > providing a callback solution lets people engineer their own prefix ma= tching<br> > solution if that's what they need.<br> <br> Yep, that's why I like Antoine's callback suggestion.<br></blockquo= te><div><br></div><div>So are you suggesting dropping even the ensure_eager= convenience argument?<br></div></div></div> --94eb2c1970aa1fd30a05540d8130-- --===============9099135998355608205== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Import-SIG mailing list [email protected] https://mail.python.org/mailman/listinfo/import-sig --===============9099135998355608205==--