Re: m4p: an implementation of GNU m4 in Python
Nikolaos Chatzikonstantinou <[email protected]> Mon, 12 May 2025 09:54:54 -0500
| Newsgroups | gmane.comp.gnu.m4.general |
|---|---|
| Message-ID | <CAAQmekeaSw=CyABgLmXgsEh6kqWa2z4ZJvGXDL2ky0pqSn0ZFw@mail.gmail.com> |
--000000000000605f1b0634f18092 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, May 12, 2025, 9:16=E2=80=AFAM Eric Blake <[email protected]> wrote: > [Dropping automake; at this point, m4-discuss is the best place to > discuss how m4p is progressing] > > On Sat, May 10, 2025 at 04:45:26PM -0400, Nikolaos Chatzikonstantinou > wrote: > > > Try this in the source dir (one time only): > > > > > > python3 -m venv .venv > > > source .venv/bin/activate > > > pip install -U pip setuptools wheel > > > pip install -e .[tests] > > > > > > Then anytime you want to try the software navigate to the source > directory > > > and do: > > > > > > source .venv/bin/activate > > > > > > To activate the environment and use "m4p" directly as a command. Any > > > changes you make to the source code should be reflected in that m4p > script. > > > > > > > And I forgot that you can run the tests with > > > > pytest > > > > And select a particular test with > > > > pytest -k define > > > > E.g. picks tests/resources/define.txt > > Thanks; I now have an 'm4p' binary to play with. > > Without $* and $@ support, tail recursion is not plausible, so I will > wait until you have that implemented. > I did not realize that tail call optimization is in effect. I'll have to do that yes! Also I should implement $* and $@. An obvious easy difference - you still don't have short-circuiting > working in eval: > > $ echo 'eval(1 || 1/0)' | m4 > 1 > $ echo 'eval(1 || 1/0)' | m4p > m4:stdin:1: divide by zero in eval: 1 || 1/0 > 1 > > and a bit less obvious is the difference between 1.4.19 and 1.4.20 on > when short-circuiting actually works (if you are aiming for a true > 1.4.19 experience, you need bug-for-bug compatibility; if you are > instead aiming for latest release compatibility, you should favor the > fixed eval): > > $ echo 'eval(1 || (1/0))' | m4-1.4.19 > m4:stdin:1: bad expression in eval (excess input): 1 || (1/0) > > $ echo 'eval(1 || (1/0))' | m4-1.4.20 > 1 > $ echo 'eval(1 || (1/0))' | m4p > m4:stdin:1: divide by zero in eval: 1 || (1/0) > 1 > I have a file where I noted some of these here: https://codeberg.org/annoyingusername/m4p/src/branch/main/tests/resources/e= val_bugs.txt At the time I thought the behavior was not right and I shelved it for later. I noticed the changelog for m4 1.4.20 and I think I can try for the byte for byte compatibility for that version instead. Another obvious difference that I depend on (and POSIX says it should > work): > > $ echo foo | m4 -Dfoo=3Dbar > bar > $ echo foo | m4p -Dfoo=3Dbar > usage: m4p [-h] [-P] [-v] [-I DIRECTORY] [--debugfile [FILE]] [FILE ...] > m4p: error: unrecognized arguments: -Dfoo=3Dbar > I will add this eventually too, after I get debugmode (the final macro) flags to work. And then there's this stress test that's worth handling gracefully > (you can't prevent stack overflow, because that's computationally > equivalent to solving the Halting Problem, so the best you can do is > gracefully handle it rather than spewing thousands of lines of errors > at the user when they write a script with poor recursion): > > $ echo 'define(a,a(a))a' | m4; echo $? > m4: stack overflow > 1 > $ echo 'define(a,a(a))a' | m4p; echo $? > m4: stack overflow > Traceback (most recent call last): > File "/home/eblake/m4p/.venv/bin/m4p", line 8, in <module> > sys.exit(main()) > ^^^^^^ > File "/home/eblake/m4p/m4p/main.py", line 27, in main > _main() > File "/home/eblake/m4p/m4p/main.py", line 102, in _main > parser.parse() > File "/home/eblake/m4p/m4p/parser.py", line 511, in parse > args =3D self.consume_arguments() > ^^^^^^^^^^^^^^^^^^^^^^^^ > ... ~2500 lines later... > RecursionError: maximum recursion depth exceeded > 120 > Thanks, I will definitely handle that exception. PS. I'm tracking feature progress here, https://codeberg.org/annoyingusername/m4p/wiki/Features and I will note some of the things you mentioned when I get back home. Regards, Nikolaos Chatzikonstantinou > --000000000000605f1b0634f18092 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><div class=3D"gmail_quote gmail_quote_container"><di= v dir=3D"ltr" class=3D"gmail_attr">On Mon, May 12, 2025, 9:16=E2=80=AFAM Er= ic Blake <<a href=3D"mailto:[email protected]">[email protected]</a>>= wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8= ex;border-left:1px #ccc solid;padding-left:1ex">[Dropping automake; at this= point, m4-discuss is the best place to<br> discuss how m4p is progressing]<br> <br> On Sat, May 10, 2025 at 04:45:26PM -0400, Nikolaos Chatzikonstantinou wrote= :<br> > > Try this in the source dir (one time only):<br> > ><br> > > python3 -m venv .venv<br> > > source .venv/bin/activate<br> > > pip install -U pip setuptools wheel<br> > > pip install -e .[tests]<br> > ><br> > > Then anytime you want to try the software navigate to the source = directory<br> > > and do:<br> > ><br> > > source .venv/bin/activate<br> > ><br> > > To activate the environment and use "m4p" directly as a= command. Any<br> > > changes you make to the source code should be reflected in that m= 4p script.<br> > ><br> > <br> > And I forgot that you can run the tests with<br> > <br> > pytest<br> > <br> > And select a particular test with<br> > <br> > pytest -k define<br> > <br> > E.g. picks tests/resources/define.txt<br> <br> Thanks; I now have an 'm4p' binary to play with.<br> <br> Without $* and $@ support, tail recursion is not plausible, so I will<br> wait until you have that implemented.<br></blockquote></div></div><div dir= =3D"auto"><br></div><div dir=3D"auto">I did not realize that tail call opti= mization is in effect. I'll have to do that yes! Also I should implemen= t $* and $@.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></div><= div dir=3D"auto"><div class=3D"gmail_quote gmail_quote_container"><blockquo= te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so= lid;padding-left:1ex"> An obvious easy difference - you still don't have short-circuiting<br> working in eval:<br> <br> $ echo 'eval(1 || 1/0)' | m4<br> 1<br> $ echo 'eval(1 || 1/0)' | m4p<br> m4:stdin:1: divide by zero in eval: 1 || 1/0<br> 1<br> <br> and a bit less obvious is the difference between 1.4.19 and 1.4.20 on<br> when short-circuiting actually works (if you are aiming for a true<br> 1.4.19 experience, you need bug-for-bug compatibility; if you are<br> instead aiming for latest release compatibility, you should favor the<br> fixed eval):<br> <br> $ echo 'eval(1 || (1/0))' | m4-1.4.19<br> m4:stdin:1: bad expression in eval (excess input): 1 || (1/0)<br> <br> $ echo 'eval(1 || (1/0))' | m4-1.4.20<br> 1<br> $ echo 'eval(1 || (1/0))' | m4p<br> m4:stdin:1: divide by zero in eval: 1 || (1/0)<br> 1<br></blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto"= >I have a file where I noted some of these here: <a href=3D"https://codeber= g.org/annoyingusername/m4p/src/branch/main/tests/resources/eval_bugs.txt">h= ttps://codeberg.org/annoyingusername/m4p/src/branch/main/tests/resources/ev= al_bugs.txt</a></div><div dir=3D"auto"><br></div><div dir=3D"auto">At the t= ime I thought the behavior was not right and I shelved it for later. I noti= ced the changelog for m4 1.4.20 and I think I can try for the byte for byte= compatibility for that version instead.</div><div dir=3D"auto"><br></div><= div dir=3D"auto"><div class=3D"gmail_quote gmail_quote_container"><blockquo= te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so= lid;padding-left:1ex"> Another obvious difference that I depend on (and POSIX says it should<br> work):<br> <br> $ echo foo | m4 -Dfoo=3Dbar<br> bar<br> $ echo foo | m4p -Dfoo=3Dbar<br> usage: m4p [-h] [-P] [-v] [-I DIRECTORY] [--debugfile [FILE]] [FILE ...]<br= > m4p: error: unrecognized arguments: -Dfoo=3Dbar<br></blockquote></div></div= ><div dir=3D"auto"><br></div><div dir=3D"auto">I will add this eventually t= oo, after I get debugmode (the final macro) flags to work.</div><div dir=3D= "auto"><br></div><div dir=3D"auto"><div class=3D"gmail_quote gmail_quote_co= ntainer"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex"> And then there's this stress test that's worth handling gracefully<= br> (you can't prevent stack overflow, because that's computationally<b= r> equivalent to solving the Halting Problem, so the best you can do is<br> gracefully handle it rather than spewing thousands of lines of errors<br> at the user when they write a script with poor recursion):<br> <br> $ echo 'define(a,a(a))a' | m4; echo $?<br> m4: stack overflow<br> 1<br> $ echo 'define(a,a(a))a' | m4p; echo $?<br> m4: stack overflow<br> Traceback (most recent call last):<br> =C2=A0 File "/home/eblake/m4p/.venv/bin/m4p", line 8, in <modu= le><br> =C2=A0 =C2=A0 sys.exit(main())<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^^^^^^<br> =C2=A0 File "/home/eblake/m4p/m4p/main.py", line 27, in main<br> =C2=A0 =C2=A0 _main()<br> =C2=A0 File "/home/eblake/m4p/m4p/main.py", line 102, in _main<br= > =C2=A0 =C2=A0 parser.parse()<br> =C2=A0 File "/home/eblake/m4p/m4p/parser.py", line 511, in parse<= br> =C2=A0 =C2=A0 args =3D self.consume_arguments()<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^^^^^^^^^^^^^^^^^^^^^^^^<br> ... ~2500 lines later...<br> RecursionError: maximum recursion depth exceeded<br> 120<br></blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"aut= o">Thanks, I will definitely handle that exception.</div><div dir=3D"auto">= <br></div><div dir=3D"auto">PS. I'm tracking feature progress here, <a = href=3D"https://codeberg.org/annoyingusername/m4p/wiki/Features">https://co= deberg.org/annoyingusername/m4p/wiki/Features</a> and I will note some of t= he things you mentioned when I get back home.</div><div dir=3D"auto"><br></= div><div dir=3D"auto">Regards,</div><div dir=3D"auto">Nikolaos Chatzikonsta= ntinou</div><div dir=3D"auto"><div class=3D"gmail_quote gmail_quote_contain= er"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef= t:1px #ccc solid;padding-left:1ex"> </blockquote></div></div></div> --000000000000605f1b0634f18092--