Re: m4p: an implementation of GNU m4 in Python

On 5/9/25 23:28, Nikolaos Chatzikonstantinou wrote:
> On Fri, May 9, 2025 at 11:13 PM Jacob Bachmeyer<[email protected]> wrote:
>> On 5/9/25 16:45, Nikolaos Chatzikonstantinou wrote:
>>> [...] I have not had any
>>> benchmarks, but from roughly looking at
>>> how long tests take I'm measuring a 100x slowdown.
>> Have you tried running it with PyPy? (<URL:https://pypy.org/>)
> No, I'm using CPython. Some of the slow down certainly has to do with
> my use of bytestrings (allocated even for trivial things like
> read-only slices). Another is that Python itself needs to load before
> the program runs, and the unit tests are small programs, so most of
> the time is spent doing that. I wouldn't care too much to optimize the
> Python version as it's meant to be didactic of how an m4
> implementation would look like.

My understanding (possibly incorrect) is that PyPy is a "drop-in" 
alternative for CPython.  The most you might have to do is ensure that 
m4p works in the version of Python that PyPy implements.

You may be right about unit test overhead (and some programs actually 
run *slower* under PyPy as compared to CPython), but the only way to be 
sure is to try it.

Would it not be helpful if the "didactic m4" were also quite usable?

>>>    I'm hoping to
>>> rewrite it in Rust later to address that.
>> Unfortunately, Rust has ... problems as a language and ecosystem.  For
>> reasons that are partially technical and partially political, the
>> general view of Rust at the GNU project seems rather dim to me.  (An
>> "AI"-assisted switch to Rust was the April Fool's joke earlier this year.)
>>
>> The political problems were bad enough to spawn a Rust fork
>> (<URL:https://crablang.org/>) which amusingly has provided most of the
>> logo for the ongoing efforts to implement a Rust frontend for GCC.
>> (<URL:https://rust-gcc.github.io/>)
>>
>> Rust should probably be considered unusable for GNU development until
>> that GCC frontend is complete due to one of the larger problems with
>> Rust:  the Rust language drifts "out from underneath" existing code.
> What does it mean to drift out from underneath?

Programs either no longer compile with a newer compiler or (worse) 
compile but with different semantics.  I *think* Rust has only had the 
first of those problems, but have not followed the issue closely enough 
to be sure.  Python infamously had the latter problem with the 
Python2->Python3 transition, but that at least had a major version bump 
associated with it.

>> That is not acceptable at GNU, but the GCC Rust-language frontend is
>> expected to follow the same convention GCC uses for other languages
>> where the standards have evolved with time.
>>
>> I believe that their first goal is to implement Rust 1.49, which gccrs
>> will continue to support into the indefinite future.
> I'm not aware of this. I couldn't find good information online; I
> decided to look up Richard Stallman's talks to see if he mentions
> Rust, and in 21:19-23:16 of
> <https://techrights.org/o/2023/03/17/rms-talk-for-latest-perils/> he
> mentions the trademark copyright from the Rust foundation as his only
> issue and that he has no other opinion of Rust. Is this then the
> issue? Does it have to do with cargo? I could write this all in core
> Rust avoiding other packages.

There was some controversy over the Rust Foundation asserting trademark 
rights well beyond what is normally considered the scope of trademarks.  
Some of their claims, if valid, would have effectively made rustc 
non-free despite its Free copyright license terms.

I am not familiar with cargo, but fewer dependencies are almost always a 
good thing.  The major risk I see is the potential depth of the 
dependency graphs.

> In any case, I can see a few paths moving forward:
>
> 1. Keep working on the Python fork as an alternative to those who have
> issues with GNU m4 (like compilation issues) or want to educate
> themselves on an m4 implementation.
> 2. Rewrite in Rust 1.49 so that when gccrs is up to speed it can be
> incorporated with the GNU project.
> 3. Rewrite in C++, or GNU Guile, or even another language.
> 4. Rewrite in modern Rust, and do not try to incorporate it with GNU
> (it's still free software).

Of those, I personally think that the Python variant is most likely to 
be useful as an educational tool.  GNU m4 is written in C and is likely 
to build on any system that can build CPython (which I believe may also 
be a bootstrap step for PyPy).

While the most recent GNU m4 release was 4 years ago, development in the 
Git repository continues; the most recent commits are from 3 weeks ago.  
In fact, the commit logs suggest that a 1.4.20 release is upcoming. 
(<URL:http://git.savannah.gnu.org/cgit/m4.git/log/?h=branch-1.4>)


-- Jacob