RE: Barracuda: Build system modifications

Hi Christian,

see comments below...

At 01:50 PM 1/3/2003 -0700, you wrote:
>Hi Jake,
>
> > That being the case, the Barracuda build will *not* suit the needs
> > of developers requiring a template for a proprietary build where
> > they want to hide most things.
>
>I agree with this; what I would like, however, is to keep the build very
>similar (compatible in structure) to a proprietary build model. We have one
>that works very nicely which I hope to contribute soon, which might help
>clarify things (more details below).

A webapp build structure, I presume.  Not everything is a webapp.


> > I wanted to remedy this by moving the deployment root to a "build"
> > directory which is created at build time.
>
>That's the part where I lose you - I'm not seeing any advantage to moving
>the deployment root to someplace else (or at least, I'm not seeing any
>advantage that can't be solved by other means, while still keeping the
>deployment root at the top, and having the whole system prebuilt on
>checkout).

Well, throw out all other reasons and just use this one simple reason:

<target name="clean">
<delete dir="build"/>
</target>

It doesn't get much simpler than that to do a full clean.  Also, if you 
don't do a full clean, you can end up resources, classes, and jars that 
shouldn't be in the build.  For instance, say you built something or copied 
something, then you rename it on the src side of things.  The original 
copied resource still exists where you copied it and the only way you are 
going to make sure the system is fully synced up is to do a full clean 
build.  Just counting on copying instead of doing a clean is 
problematic.  I *always* do a clean.  I don't care if the system is 
pre-built on checkout.  I'm going to do a clean build anyway.


> > Well, if we created a "src" directory that contained other
> > specific source directories underneath, you not have the root
> > crowding problem.  This is soved, anyway, by separating the
>
>Well, in one sense, yes. But now your build system can't just assume that
>everything under src is to be compiled (ie. what's under src is not really
>src...its other subdirectories, and the src is under those). So I'm not sure
>that's a real gain.

It is purely organizational and the build would know what source 
directories are for compiling and which are scripts, and whatnot.  Like I 
said, organization of the src directory that I laid out is completely 
debatable.  I was just showing how it solves the "crowding" issue you 
mentioned.

>So it sounds like the real issue is that you don't like seeing the src under
>WEB-INF? I guess I'm not seeing the problem with that (at least if you don't
>need to make your src explicitly accessible).

It forces the semantics of what your app is all about.  If you put it under 
WEB-INF, well, you app is quite obviously a "webapp".  However, if I am 
going to have a generalized build template for all kinds of projects, I 
don't want to force, these semantics.  It also defeats the purpose of the 
original intent of Baracuda being an open source project where the source 
is fully viewable by the public.

>Now perhaps where you are going with this is that you are thinking when you
>deploy a production app, you don't want to even put the src on the server.
>I'd agree with this - for our production apps (distinct from a "framework"
>app like Barracuda) we use a dual-webapp structure. Here's an example:
>
>/MyApp
>     /MyAppDeploy
>         /WEB-INF
>             /lib (not under cvs)
>             /lib-cvs
>     /MyAppDev
>         /docs
>         /WEB-INF
>             /bin
>             /src
>             /src-mockups
>             /lib (not under cvs)
>             /lib-cvs
>
>In this approach you do all your development under MyAppDev; then when
>things are solid we jar and tag the MyAppDev tree (so we can pull any given
>build as needed). We then "deploy" into MyAppDeploy, copying only what is
>needed for the actual production install (ie. just jars usually). We then
>tag the MyAppDeploy tree with our production release info. Now on a
>production box, we "deploy" a version of the app by checking out the
>appropriate tag from cvs, and asking the app to "install" itself (copying
>jars from lib-cvs to lib). We then cycle tomcat and everything is golden.

This is almost exactly the concept of the "build" directory except that, 
given the fact that you never delete MyAppDeploy, you may have left over 
cruft.  I've had cases where I think everything is just fine.  My app 
bulids fine at build time and breaks at runtime.  Then I just decide to do 
a clean build.  And, what do you know, my app runs just fine.  Ant is 
great, but there are lots of small issues that you can't always predict 
when doing builds.  A clean build is the only sure solution.

Either way, you can always deploy to your server without ever recycling the 
server at all and without worrying about locked jars in your build 
directories.  In Barracuda currently, you can do one of a couple things:

this is for development purposes
ant install

This is for production purposes (this one needs a few tweaks, but I know 
exactly what to do to make it robust and flexible, I just haven't had the 
need to yet)
ant deploy

Alternatively, with Tomcat, you can set up a build which build Tomcat 
around a custom installation of Tomcat using CATALNIA_BASE.  You can 
provide separate server config files and everything.  David Li's 
xmlc-2.2alpha example app provides this functionality.  He also builds to a 
"build" directory for easy cleans.
http://homepage.mac.com/taweili/XMLC.html


>Now, we're using this approach in production apps, and it works very well.
>But the whole dev vs. deploy mechanism hasn't really proven necessary for
>our frameworks (we have additional ones beside Barracuda) - there its been
>sufficient to simply have a std dev tree, like Barracuda does (although we
>keep our source under WEB-INF, rather than the top level directory). My
>point in all this is just that the approach I'm talking about seems to solve
>your problems at least as well as your proposal does, plus it uses a similar
>structure for "framework" projects and for "end application" projects (which
>have the additional deploy webapp tree). But both of them are ready to run
>out of the box, and both keep the webapp root at the top of the project.

I still don't like the semantic stranglehold you put on your builds by 
putting src in WEB-INF.  Again, not everything is a webapp and I don't want 
it deployed until I am ready anyway and that *always* means doing a clean 
build, IMO.

>Ok, I'm running out of time (have a meeting to go to), but I'll try and get
>a couple more comments in before I have to run out the door.
>
> > I don't think that a "build" directory generated in the root of
> > the project is considered "buried somewhere down in the bowels of
> > the system".  It is sitting right there.  Now, if you are talking
> > about the contrib projects, I solved that mostly by creating the
> > build-contrib.xml which allows you to build all the contrib
> > projects from the root level even though they are physically
> > built "down in the bowels of the system".  I am fine with moving
> > the contrib projects up higher in the directory structure.  I was
> > just working within the constraints of the existing build system.
> >  I think I came up with a pretty nifty solution given the
> > constraints, IMO.
>
>Ok, so I'd like to look more closely at the contrib stuff to see what you've
>done there. I will try to do that sometime soon. In the meantime, can you
>give us a quick high level overview of what it does and how it works? (ie.
>what do you see as the advantages of it)

Haven't you every tried Diez' app?

Make sure CATALINA_HOME is set, make sure Tomcat is running and has xmlc 
related jars in CATALINA_HOME/shared/lib, and make sure you have set the 
manager username and password in build.properties to an existing user 
(conf/tomcat-users.xml) that has the role of "manager".

Now type:

ant contrib -Dsubproject=all -Dsubtarget=install

After that succeeds, go to:

http://localhost:8080/contrib-dbroggisch/

I don't know what you mean by "advantages"?  The fact that Diez' project 
exists provides a useful example of how to do form mapping in Baracuda and 
shows how to utilize his contrib package.  Other than that, please explain 
what you mean by "advantage".


> > Because contrib projects contain both general use libraries which
> > ought to be generally available and can be built within the
> > normal Barracuda build *and* contributors have useful webapps to
> > show off functionality of their libraries.  In addition, some
> > contrib projects might only consist of a webapp using the normal
> > Barracuda libraries, but also have some unique and useful
> > techniques to show off.
> >
> > BConfig and Admin apps are unique in that they can be built
> > within the confines of the Barracuda system itself.  They are
> > really sub-applications, though.  Not fully separate webapps.
> > The concept is, actually, quite inventive and I commend you for
> > creating such a flexible concept of a movable app, static
> > resources and all.  However, most times, webapps have too many
> > unique things to just be in a .jar file.  They need their own
> > separate webapp infrastructure from the web.xml to custom
> > libraries...all the way to a custom build.  Mixing these with the
> > current Barracuda build would just make it a big mess.  The
> > separation allows flexibility from everyone.
> >
> > Now, we could make these separate CVS modules.  I have no problem
> > with that.  However, that wasn't possible when we were at e.org,
> > so I made due with what we had to work with...building contrib
> > webapps under the current source build structure.  Also,
> > currently you can build and deploy (install) all the contrib
> > projects with one command:
> >
> > ant contrib -Dsubproject=all -Dsubtarget=install
> >
> > If Tomcat is running, every existing contrib project which has a
> > "webapp" folder in it will get installed in memory to Tomcat.  If
> > we move each contrib project to its own module, we should write
> > some mechanism to do these installations in a similar way.
>
>Ok...I'm out of time, and there's too much here for me to respond too. I
>will try and finish  this thought process up tonight. Thanks for taking the
>time to respond!
>
> > Either way, if you want this out the door today with a decision
> > made on the build system, I'm afraid we might just have to defer
> > to your wisdom.  I'm just curious how the existing contrib
> > infrastructure fare.
>
>Ack! Don't just trust my wisdom; you've got good opinions too! I think what
>we need to do is just make sure we more thoroughly explore the requirements
>here to figure out the best solution for both of us. I'm sure we'll come up
>with something that's the best of both!

Unfortunately, I have mostly run out of time myself and am going to have to 
defer to your wisdom.  We can always change it later, I guess.  My worry is 
that you don't have any idea of how the contrib project build system 
works.  We need to incorporate those somehow.  I want to get the server up 
and running as well.  It seems as if it is going to get up faster if I just 
wait and see what you come up with.  Maybe I'll like it just fine in the 
end?  Anyhow, I've spent too much of my day off sitting in front of a 
computer thinking about this and won't have much time this weekend or next 
week to think about this.  I officially defer to you.   As long as 
everything works as it did before, albeit in a different build structure, 
I'll be happy.  I'll probably keep doing my own builds the way I've 
described, though.

later,

Jake

>Ok, I'm out of here! Talk to you later!
>Christian
>
>_______________________________________________
>Barracuda mailing list
>[email protected]
>http://www.enhydra.org/mailman/listinfo.cgi/barracuda
>FAQ - http://www.jguru.com/faq/Barracuda