Idea: concept of a builder or transformer to compliment loaders

--===============1733621792==
Content-Type: multipart/alternative; boundary=001a1134ae7c5d94dc051725c7ad

--001a1134ae7c5d94dc051725c7ad
Content-Type: text/plain; charset=UTF-8

I should start off by saying I don't plan to pursue this idea, but I wanted
to write it down for posterity and in case anyone else has thought about
this.

That being said, the idea of macros and other source-transforming things
done to Python code has come up a few times on python-ideas as of late. Now
experimenting with this sort of thing using a custom loader is not hard,
and thanks to importlib.abc.ResourceLoader.source_to_code()
<https://docs.python.org/3/library/importlib.html#importlib.abc.InspectLoader.source_to_code>
it's fairly easy to do (by design; I tried to initially structure
importlib's APIs to making alternative storage backends easy as well as
alternative syntax stuff like Quixote from back in the day).

But one thing I realized is that while finders and loaders are necessary
for alternative code storage mechanisms, they are not the right abstraction
for tweaking code semantics. Really all you need is a function that takes
in source code and spits out a code object to use with exec() (hence
ResourceLoader.source_to_code() even existing). It somewhat sucks that
people who just want to tweak code semantics have to define a loader
subclass and instantiate a new finder when all that is mostly stuff that
doesn't concern them. It also sucks that they would have to do that for
every storage type, e.g. local files and zip files.

Now I don't have a solid solution to propose for this niche use case. It
makes me want to have some kind of way to register compiler functions, but
that would be limiting if it went source -> code object. AST -> AST would
allow for chaining much like Victor has proposed in the past, but it also
means that people who want a transpiler to go source -> source are left
out. And then there is the whole thing of how to get the loaders to know of
these transpilers/transformers/compilers as adding more global state to sys
feels dirty (maybe an attribute on finders that they can draw from if they
so choose?), but maybe it isn't that big of a deal as long as they are just
callables and people realize they must be re-entrant.

As I said, I don't plan to work on this, but I wanted to get my ideas
written down in case someone else cared.

--001a1134ae7c5d94dc051725c7ad
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I should start off by saying I don&#39;t plan to pursue th=
is idea, but I wanted to write it down for posterity and in case anyone els=
e has thought about this.<div><br></div><div>That being said, the idea of m=
acros and other source-transforming things done to Python code has come up =
a few times on python-ideas as of late. Now experimenting with this sort of=
 thing using a custom loader is not hard, and thanks to <a href=3D"https://=
docs.python.org/3/library/importlib.html#importlib.abc.InspectLoader.source=
_to_code">importlib.abc.ResourceLoader.source_to_code()</a> it&#39;s fairly=
 easy to do (by design; I tried to initially structure importlib&#39;s APIs=
 to making alternative storage backends easy as well as alternative syntax =
stuff like Quixote from back in the day).</div><div><br></div><div>But one =
thing I realized is that while finders and loaders are necessary for altern=
ative code storage mechanisms, they are not the right abstraction for tweak=
ing code semantics. Really all you need is a function that takes in source =
code and spits out a code object to use with exec() (hence ResourceLoader.s=
ource_to_code() even existing). It somewhat sucks that people who just want=
 to tweak code semantics have to define a loader subclass and instantiate a=
 new finder when all that is mostly stuff that doesn&#39;t concern them. It=
 also sucks that they would have to do that for every storage type, e.g. lo=
cal files and zip files.</div><div><br></div><div>Now I don&#39;t have a so=
lid solution to propose for this niche use case. It makes me want to have s=
ome kind of way to register compiler functions, but that would be limiting =
if it went source -&gt; code object. AST -&gt; AST would allow for chaining=
 much like Victor has proposed in the past, but it also means that people w=
ho want a transpiler to go source -&gt; source are left out. And then there=
 is the whole thing of how to get the loaders to know of these transpilers/=
transformers/compilers as adding more global state to sys feels dirty (mayb=
e an attribute on finders that they can draw from if they so choose?), but =
maybe it isn&#39;t that big of a deal as long as they are just callables an=
d people realize they must be re-entrant.</div><div><br></div><div>As I sai=
d, I don&#39;t plan to work on this, but I wanted to get my ideas written d=
own in case someone else cared.</div></div>

--001a1134ae7c5d94dc051725c7ad--

--===============1733621792==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Import-SIG mailing list
[email protected]
https://mail.python.org/mailman/listinfo/import-sig

--===============1733621792==--