Re: [RFC] improving support for building native tools in cross setups

On Thu, Jan 30, 2014 at 01:33:17PM +0000, Gavin Smith wrote:
> On Wed, Jan 29, 2014 at 10:40 PM, Yann Dirson <[email protected]> wrote:
> > Hello,
> >
> > It is not uncommon for software packages to build tools to be executed
> > at build time, to generate data files or more input files to compile.
> >
> > Unless I miss something, the current autotools does not help much
> > dealing with this for cross-compilation:
> >
> 
> (Link to discussion on other mailing list for reference:
> http://lists.gnu.org/archive/html/automake/2014-01/msg00006.html.)
> Using hand-written rules for these data files and tools with
> AX_CC_FOR_BUILD is a solution if the tools are written in C. If that
> works, even though it needs hand-written rules, unless this is a
> common problem it's probably better to do that then complicate the
> autotools (unless the extra features definitely won't get in the way
> when they're not being used), given that at present it's assumed that
> all files are being built for the same architecture. It would have to
> be thoroughly thought out and tested.

The problem is widespread enough, that tools like scratchbox
(http://scratchbox.org/) have been created, to workaround deficient
build systems by running such tools emulated by qemu.  Clever hack, I
admit, but a huge waste of resources.  A well-thought solution
directly integrated in the build system will be cleaner.

AX_CC_FOR_BUILD, AX_PROG_CXX_FOR_BUILD and such are still IMHO in the
realm of workarounds: for proper support we'll need to write one such
macro for each supported language, and for various binutils,
essentially cut'n'pasting the existing ones.  Finding a clean way to
write a single macro fitting both needs and avoiding duplication would
be just great.  But the first point to be dealt with is IMHO the
automake side: as AX_CC_FOR_BUILD shows we'll be able to do something
anyway for autoconf, without necessarily modifying it.

I had myself made a hack for gnugo some time ago[1], just using
automake features, and relying on the openembedded toolchain
explicitely passing paths to native tools, so no autoconf-level work
was attempted at that time.  As you can see, such a patch can be huge.

[1] http://lists.gnu.org/archive/html/gnugo-devel/2010-09/msg00004.html


> > For the first point I was able to derive an AC_PROG_BUILD_CC from
> > AC_PROG_CC, to probe for a compiler using AC_CHECK_PROG instead of
> > AC_CHECK_TOOL, and set BUILD_CC.  That's just a PoC, not sufficient in
> > itself, since other tools from the toolchain will be required (and it
> > partly duplicates code, which is not nice eiher).  At least it does
> > not *require* modificaiton to autoconf in itself, as the macro can be
> > embedded into source packages if needed.
> >
> > For the second point, it would seem a good idea at some point to
> > support a "native_" prefix, so setting eg. "native_PROGRAMS" will
> > trigger the use of BUILD_* tools.
> >
> 
> The "noinst_" prefix could be closely related. It would seem that most
> of the time you wouldn't want to install these files.

Right, and the reverse is usually true as well, most of noinst_ progs
I know of are really meant for the build machine.  Wonder if there
will be any use left for noinst_ if we end up implementing native_ :)

> (PS What's PoC? Piece of cake?)

Proof on concept.


> > But as a first step I experimented with setting $program_CC, like can
> > be done for flags.  The PoC patch below attempts to do that, and in
> > fact setting "gnushogi_CC=$(BUILD_CC)" does produce the expected
> > result in Makefile.in (separate rules for gnushogi-*.o are produced,
> > with the correct CC being called).
> 
> File-specific variables could work. I imagine you are using generic
> rules for cross-compilation, and explicit rules for when you are
> building your auxiliary tools?

Just using generic rules for everything till now, with just per-prog
CC and my patch to differentiate the rule.

> My general advice is to keep rules, flags, variables, etc. used for
> building files for one architecture completely separate from those
> used for another architecture. Also you have to make sure that you
> find all the tools you need.

Flags and many variables will have to be separated, yes - there is no
reason why a compiler of the same family would be used for --host and
--build.

But rules generated by automake are generic in structure, supporting
all kinds of compilers and flags, native as well as cross settings,
etc.  This is why I took the direction of not duplicating the rules...


> > Where I'm running into problems is that setting CCLD in the same way
> > requires further changes (eg. expanding LINK for all targets instead
> > of using this as a variable, or overriding LINK instead which looks
> > not as clean).
> >
> 
> I don't know if you saw in the manual that you can set a
> target-specific linker? See "maude_LINK" at

... and yes, I have seen I can override _LINK, but overriding it means
again cut'n'paste, and would tie a Makefile.am too much to the
automake version.


> > And without even trying to touch this, I also have
> > another problem to deal with, where requesting my native binary
> > triggers some sort of default rule attempting to build the binary in
> > one step directly from the C source, with the cross-cc and wrong flags.
> >
> 
> If it is running the wrong rule make sure there is an explicit rule
> for the target in the makefile.

Yep, going on a hunt :)

Best regards,
-- 
Yann