Re: Guile support in GNU make

On Wed, 2011-09-21 at 21:04 +0200, Boris Kolpackov wrote:
> This made me think that perhaps we could do something similar for GNU 
> make. That is provide a way for a makefile to request the loading of a
> plugin module. If the module does not exist, make can try to build it
> and re-exec in exactly the same way included makefiles are handles right
> now (this will allow the second category of users mentioned about to
> distribute source code for their GNU make plugins together with their
> software). 
> 
> Once loaded, a plugin is given a chance to register additional functions.
> Maybe we can also define some additional hooks, like when up-to-dateness
> of a file is determined, or before a command is executed. But I think just
> extra functions will be a good first step.
> 
> The nice thing about the GCC plugin architecture is that the plugin
> is loaded into the GCC address space. This means that it can access
> all the functions, data structures, etc., that built-in GCC code can
> access. I think the same approach will work fairly well for GNU make
> as well.

This is actually the part that worries me the most about this idea.  As
you're aware the internals of GNU make are in no way clean and
protected: global variables abound and there are few APIs internally
that allow for ease-of-use.  I would be very nervous about exposing that
to user-written code, without any control.  I guess if it breaks they
get to keep both pieces, but still.

Second, I wonder how portable this capability really is.  It works well
on Linux but how easy is it to get similar behaviors on the other
POSIX-based systems, much less Windows etc.?

Still it's a cool idea that make would build its own loadable module and
then load it :-).  Making this easy might be complex though: ideally
you'd like to add some more builtin variables that provide the rules for
building a loadable object on the local system, based (I guess) on
configure-discovered commands.  Would this require libtool, which I
really hate?  Or could it be done more simply?  I don't know.  Some
systems (I'm looking at you, AIX) seem to go out of their way (IIRC,
it's been a while) to make building a dynamic object tricky.

> And if we have something like this, it will be fairly straightforward
> to add Guile support as just another plugin.

Actually from what I've seen I think it will be difficult to add Guile
support in this way: because Guile is garbage-collected you have to play
some tricks with your program's main() so that Guile can understand how
the stack is used/ordered.  I'm not sure how possible it is to bolt on
Guile support solely through a dlopen()'d shared object.