Re: Re: software design clarification

On Tue, May 08, 2007 at 12:23:41PM -0400, Stefan Seefeld wrote:
> Nathaniel Smith wrote:
> > On Fri, May 04, 2007 at 07:39:56AM -0400, Stefan Seefeld wrote:
> >> Yes, Elsa is definitely on my radar. I have been looking at the code, as
> >> well as the docs, for a while, and even had some discussion with its main
> >> author.
> >>
> >> What holds me back from hooking that up as a C++ parser is principaly
> >> two reasons:
> >>
> >> * It doesn't have all the features Synopsis supports, and thus, even though
> >>   it is more complete, it would cut on the functionality we can provide.
> > 
> > Hmm, what is it missing, out of curiosity?  I know it has full
> > tracking of the relation between source and AST, and my impression
> > is the type system is complete-ish... I guess it probably doesn't
> > parse comments but that's hardly a showstopper...
> 
> Right, if it was only the comments...
> It was my impression that Elsa wouldn't store pointers into the source buffer,
> thus making it impossible to transform the source code (as opposed to regenerate
> it). In other words, can elsa provide lossless source-to-source transformation ?

I might be missing something in the distinction between "transform"
and "regenerate", but the Elsa AST does know enough about the
correspondence between AST and source byte ranges that it's being used to
generate (clean, acceptable-to-reviewers) patches against Mozilla right
now.  It sounds like there are some issues dealing with CPP, and Elsa
not having as complete information as one would like... OTOH, it also
sounds like Taras Glek has gotten fed up with this and is fixing it:
  http://blog.mozilla.com/tglek/2007/05/11/cpp-strikes-back/

The background is that the Mozilla team has a problem: they've
inherited a codebase that was originally written by a few hundred
full-time hackers over many years, and as a result is ridiculously
large and overgrown, to the point that they simply don't have the
manpower to deal with all of it.  So they're investing heavily in
developing automated tools: from variable renaming to dead code
removal to pointless-inheritance-hierarchy collapsing to
pointless-COM-glue removal to converting error codes to exceptions to
converting (limited chunks of) C++ to JS.  Oh, and getting good tools
to inspect the source, automatically validate high-level invariants,
with a scripting language interface...  They're totally serious about
all this, and Taras is apparently the guy working on it full time:
  http://blog.mozilla.com/tglek/
(Other interesting background links:
  http://weblogs.mozillazine.org/roadmap/archives/2006/10/mozilla_2.html
  http://weblogs.mozillazine.org/roadmap/archives/2006/11/oinkbased_patch_generation.html
)

So I bring this up because even if Elsa is missing things right now, I
think it's safe to say that they will or could be fixed soon-ish, and
there's a lot of opportunity here to combine efforts and get mutual
benefit -- esp. since both projects seem to be manpower limited and
having trouble reaching a self-sustainable development process.

(How hard is adding accurate source tracking, even if you start from
nothing?  All you have to do is stick coordinates in your tokens, then
just write the code to copy them out again into all your AST nodes?
Maybe a little boring, but...)

> >> Yet another alternative that hasn't come up yet is to try to enhance GCC
> >> itself, i.e. add all the hooks there to make it useable in a context like
> >> Synopsis (that's the path choosen by the gccxml folks). While there are
> >> a couple of people who try to convince me that this is a better way to spend
> >> my spare time, I think the reasoning is principally the same as for Elsa
> >> (see above).
> > 
> > That sounds even messier to me, and much harder to keep in sync with.
> > (Though the way Synopsis works, wouldn't it be very simple to have a
> > front-end that eats gccxml and produces the same Synopsis AST as a
> > real front-end would?  If such a processor could be implemented
> > quickly, that'd be very handy for bootstrapping the rest of Synopsis's
> > Cool Stuff and you could just drop in whatever front-end you end up
> > using, when it's ready?)
> 
> Right. Well, it was essentially people like Mark who suggested to work
> on GCC, to provide additional hooks to bind it to Synopsis, not necessarily
> to rewrite the Synopsis bits as part of GCC. (I guess there may even be some
> commercial interests into adding introspection capabilities to GCC...)

That's getting into a bunch of really complex internal GCC politics
though, isn't it?  Dunno if you really want to go there... I mean, if
there are commercial interests that make it happen independently,
cool, you can sponge off their work, but getting in there oneself
seems like a risky sort of thing to put on the critical path.

(Actually, speaking of commercial interests again, I am pretty sure
Mozilla is hiring, perhaps they would like to pay someone on this list
to make appropriate improvements to Elsa :-).  ...For that matter,
I wonder if Mark has thought of offering Mozilla a contract to help
with their crazy schemes...)

Meddlingly yours,
-- Nathaniel

-- 
So let us espouse a less contested notion of truth and falsehood, even
if it is philosophically debatable (if we listen to philosophers, we
must debate everything, and there would be no end to the discussion).
  -- Serendipities, Umberto Eco