Developing frameworks and applications in tandem and managing dependencies

John Elliot <[email protected]> Tue, 31 Jul 2012 17:48:04 +1000
Newsgroups gmane.comp.windows.off-topic
Message-ID <[email protected]>
I'm stuck.

I'm developing a web framework and suite of associated utilities. The
sort of things in the framework are components for talking to databases,
for doing logging, for creating documents, for parsing URLs, for
generating JSON, for session management and authentication, for
encryption, and things like that.

Then I'm developing applications that use the framework. The
applications have models and views and controllers, and use the
framework components to do their job (the framework uses dependency
injection and the applications provide the injected functionality).

Ideally the web framework stuff can be reused in other applications, and
I'm developing a number of applications that use the framework at the
same time.

I have a sort of 'top down' approach to developing my applications. The
idea is that I start work on the application and then as I find myself
needing support features from the framework I implement those features
in the framework and they are then available for use in an application.
So I'm developing the framework and the applications in tandem and at
any given time the framework only has the features that have actually
been needed by the applications, I'm not doing a big upfront design of
the web framework, rather I'm evolving the framework as necessary to
meet specific needs.

One problem is that as I implement more the framework can change
drastically and in incompatible ways. So I might factor something in to
the framework that previously wasn't there that then implies that the
framework and its facilities get used in different ways that are
incompatible with the way the framework was used previously.

On the one hand I just want to have one copy of the framework code that
gets reused in all of the applications. On the other hand when I do some
work on the framework that makes it incompatible with its previous
version I'm not going to be able to go through every application that
was written using the framework and update it in every place that it
used the now incompatible feature.

So I sort of need "versions" of my framework and then each application
is targeted at the version of the framework that it is compatible with.
So if Application A is written in January using version 0.1 of the
framework I can then write Application B in February using version 0.2
of the framework. Then when it comes time to review Application A in
March I can upgrade it to use version 0.2 of the framework as well and
revise and fix any incompatibilities that have been introduced.

What I can't decide on is how I should accomplish this with my version
control system. Let's say I have a project called 'framework' in my
subversion repository and then two other projects 'app-a' and 'app-b' in
my repository. When I want to use the framework in app-a, do I create a
'lib' subdirectory and then put in an svn externals definition to
'trunk' of framework in app-a/lib/framework? If I do that then I can
edit the framework in tandem with app-a just by editing the files in
app-a/lib/framework and then app-a and the framework will be coevolving
nice and easily. Then say I ship app-a and begin working on app-b. I
might again set up an svn externals for app-b/lib/framework in app-b and
then app-b and the framework can coevolve nicely and as I need new
features in the framework I just edit them in app-b/lib/framework and
the framework is updated in trunk.

If I just do that then I will have a problem when I svn update app-a.
Because the framework now has new and incompatible features that were
developed in tandem with app-b, and because app-a has an svn externals
reference to the latest version of the framework in 'trunk' then
app-a/lib/framework will be updated to trunk and this will cause
problems because app-a hasn't been updated to use the latest
refactorings of the framework.

Instead of setting up svn externals in app-a and app-b that point to
'trunk' of the framework instead I could setup externals definitions
that point to particular release tags. So an svn externals definition
for app-a/lib/framework might point to framework/tags/version/0.1 and an
svn externals definition for app-b/lib/framework might point to
framework/tags/version/0.2. If I do that though, then I can't develop
the framework and the applications "in tandem". That is, I can't edit
the files in app-a/lib/framework because they point to a release tag and
not to a branch or trunk of the framework. In that case I would have to
checkout the trunk of framework to its own working copy somewhere, add
in my new refactorings or features, test them, do a release of framework
to say version 0.3 and then update the svn externals of
app-a/lib/framework to point now at framework/tags/version/0.3. Doing
things this way is a fair bit of work and I don't want to fail at my
projects because my development processes are too slow or arduous.

Or maybe I could use some sort of hybrid approach, where when I'm
actively working on say app-b that I have app-b/lib/framework pointing
to framework/trunk, and then when I'm getting ready to release app-b I
first release the framework (giving it a new version number, say 0.4)
and then change the svn externals definition for app-b/lib/framework to
not point at framework/trunk any more but rather to point at
framework/tags/version/0.4.

I dunno. I'm just looking for an easy, manageable and practical way to
manage dependencies in my applications and libraries and I haven't yet
struck upon a process that I feel really works. I'm seriously
considering just having a copy-and-paste approach to the framework so I
have one copy of it directly in app-a, then another copy of it in app-b,
and then just have essentially different frameworks evolving to meet the
needs of each application and then when it comes time to create app-c,
app-d and app-e just finding a "most suitable version" to copy from
either app-a or app-b and then duplicating that in app-c/d/e and copying
and pasting my way to victory each time I need to do a new app. Doing
that strikes me as sloppy and unprofessional to the n'th degree but it
also seems like it might be a very practical approach *in the short
term*. If I can make it through the 'short term' successfully and
productively then maybe when the framework is mature (i.e. proven as
successful for app-a and app-b) then I can think about factoring it out
into its own project but I can allow it to incubate independently in
each application at first while it is immature and changing a lot.

Just wondering what people's approaches and philosophies are around
managing dependencies between libraries and the applications that use
them and would be very happy to receive pointers to proscriptive practices.

Regards,
John Elliot.