Re: Cross-dependencies and output folders and more

Bram Moolenaar wrote:
> 
> Dobes Vandermeer wrote:
> 

> > I think you described what I wanted to -- creating a full install tree
> > "somewhere" as part of the build.  I just don't want to have to type
> > "aap install" to get this effect.  I'd also want to be able to build
> > specific targets by name, and have to output appear in this same temp
> > install folder.
> 
> You could use "aap test" and have the test target invoke "install" after
> setting the destination directory.  That should be a matter of:
>
> Provided that the install target respects $PREFIX, of course.

Most likely I'll want some toplevel commands in the main.aap that say
something like ":do cptobin server" and ":do cptolib libXXX.a" so that
these will always happen, regardless of whether I run aap test or aap
install.  That way I can do manual tests without running the automated
tests.

> Thanks for the explanation.  I would summarize it with "a symbolic name
> for the directory".  

Well... I suppose you could, although you wouldn't have captured its
basic essence, or grist, as it were :).

> It could be useful.  For example, Aap already uses
> the $BDIR variable for the directory of the object files.  You wouldn't
> want to use its value on the Aap command line.  In Jam terms you would
> have a target $BDIR/foo.o called <bdir>foo.o.

Thats close; but the grist would also include your variants and the
current project (usually named after the subdir) so that you can tell
one object file from another with the same name.  As I've said, without
the "LOCATE" and "SEARCH" feature, the grist might as well be the output
folder, since the output folder is pre-determined anyway.

> I'm not sure if we need a new mechanism for this.  

Probably not; as I've said, its a "neat" feature for jam

> Including $BDIR in
> the target would be possible, so long as we can avoid the shell from
> expanding it.  This would work on Unix:
>         aap '$BDIR/foo.o'

Could work; to be honest, though, usually in jam I avoid typing the
grist on the command line anyway (since < and > are shell characters)
and define special targets inside the build file, like:

Depends install_XXX : <installed>XXX ;

And type "jam install_XXX".  This is where grist is handy, because I can
distinguish the installed version from the other version.  On the other
hand, its probably not that much effort to compute the actual output
path of <installed>XXX and but $INSTDIR/bin/XXX or whatever in that
special rule.  

This grist is a technique thats only really useful in conjunction with
SEARCH, where you specify a search path for a file and then refer to it
only by grist and name.  If the paths aren't bound late, there's no
advantage.

> Not literally ":cptobin", this sounds like you want to define an action
> with ":action cptobin" and invoke it with ":do cptobin".

OK, I see.  That should do the trick.

> For the NSIS installer, don't you always use one script to specify the
> rules?  The idea of ":produce" is that you give it the name of your
> program and a list of source files to make it from.  Aap then figures
> out how to turn each source file into an object file and link them
> together.  In this case your source files are executables or libraries
> and you only need to run NSIS to turn them into a program.  You are not
> using the mechanism that ":produce" offers to turn sources into object
> files.

Well, I suppose I forgot to include the list of files being packed into
the installer.  To be honest we currently depend the installers on
"all", and build the installers themselves infrequently.  We dont even
use the "install" target, since we only want to build installers and
dont use a build system to install our product.

> Invoking the action directly with ":do" might be what you actually want.
> You at least need to specify the files that are packed into the
> resulting executable.  A Luxury version would obtain the list of files
> from the script, something like:
> 
>         server-$version.exe build: server.nsi
>                 :syseval nsis-parser --get-files | :assing files
>                 :update $files
>                 :sys nsis $source
> 
> Then "aap build" would do everything that's needed.  You "just" need to
> write the nsis-parser command :-).

Yeah, but can I convert this into a short template, so each installer
would be a one-liner, or more importantly I can modify all installers'
behavior in one place (e.g. when fixing or replacing that template, most
likely).

> These requests are not exotic.  I think Aap should be able to support
> this.  And I don't see a reason why it couldn't, thus it is only a
> matter of someone implementing this.  Actually, the implementation is
> probably simple, the main work will be to decide how to specify a
> project with sub-projects in a nice, simple and reliable way.

Yes, definitely.  Isnt that always the hardest part?

> > - Doesnt support caching (pretty minor, though)
> 
> What kind of caching?  Aap's way of using checksums means that if you
> change a comment in a source file it will compile the source file and
> then find out the object file didn't change and the program doesn't need
> to be linked again.

Probably aap is close enough -- if it keeps outputs seperate based on
variant then your builds will still be faster.  Tools like ccache keep
old objects files so that when you revert to a previous duplicate build
(more likely because of changes to the command line rather than the
files themself) it can read the result from the cache.  The cache is a
fixed size between all your projects, so you dont have to worry about it
growing endlessly (as opposed to having many different output folders,
one for each configuration)

A good case in our system is when I turn on detailed memory tracing
using -D command line switch and then the entire tree has to be
rebuilt.  When I'm done I've usually changed just a few files and I
switch back -- now the entire tree must be built again!  In jam, this
means running jam clean because files dont depend on their command
line.  I do have different object output folders right now for release
vs debug builds, but it would be cumbersome to implement another one for
each variation of debugging defines.

> > - Uses the native shell to run commands, so you get command-line-length
> > problems, backslash vs forward slash problems (aggravated by cygwin use,
> > where you sometimes want both).
> 
> I'm afraid Aap doesn't solve this yet.  Using internal Aap commands is a
> partial solution.  I know SCons has something to work around the command
> line length limitation, we can probably use that in Aap.

Oh.. well its not hard to solve; the windows tools support reading their
command line from a file using '@'.  I think this is how everyone does
it now.

> > - Doesn't include platform-independent builtin commands (e.g. move,
> > copy, zip, unzip, see aap command list) so you have to use
> > platform-specific ones, with platform-specific flags.
> 
> That was one of the main reasons to make Aap work this way.  It's quite
> annoying having to solve "cp" versus "copy" differences in a Makefile.

Yeah, I like this about aap.

> > - Has lots of little quirks, like:
> >     - jam clean also cleans the sibling folders you depend on
> >     - sometimes you want to build a specific target, but that target has
> > a grist that is too annoying to type in.
> > - Doesn't build java with dependencies (but nobody else does, either)
> 
> This depends on the Java compiler used, it seems.  I know SCons has
> quite a bit of trouble getting it right.  Invoking the Java compiler for
> each changed source file might work (with a bit of overhead).

Well, javac allows cyclic dependencies, and no intermediate file is
generated to support this -- you just have to pass both files on the
same command line.  Also, javac may emit multiple class files per java
file, so if you wanted to copy those files around you would have to be
able to predict where they were going to end up.

Our current approach is to not allow cyclic dependencies between jar
files (a reasonable approach) and then build all the class files for a
jar, and the jar itself, at once.  This is doable and you can make the
jar file depend on all the source files.  The problem is with the
intermediate files -- if you keep them, then if you delete or rename a
class its class file wont be deleted.  If you delete the class files
then your builds are much slower :(

Thats one of the annoyances about jam I forgot to mention:
If you delete a source file, its not removed from the libraries/jars it
is in, its intermediate files aren't cleaned up, and so on.  You have to
clean it out by hand.  Would be nice to have a build system that
"remembered" what was "in there" and cleaned anything out that
disappeared.

> Subversion didn't work for me either (I started working on Subversion
> support one day and the first thing I ran into was that the FreeBSD port
> of subversion didn't work.  Still didn't get over that...).

doh! ;p

CU
Dobes


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?   SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/