Re: Re: software design clarification

Roman Yakovenko wrote:
> On 5/8/07, Stefan Seefeld <[email protected]> wrote:
>> > I am working exactly on this functionality for pygccxml project. Today
>> > it is based on GCCXML parser. I hope in few month from now, synopsis
>> > backend will be also available.
>>
>> Have you had a look at Synopsis' processor / pipeline design ? I'd
>> be curious about how useful you find it as a means to drive any source
>> transformation.
> 
> No I didn't. I promise you to find the time and to take a look on it.
> Can you
> create small list for places I should look at?

1) The tutorial section talking about scripting:
   http://synopsis.fresco.org/docs/Tutorial/scripting.html

2) Any examples using 'synopsis.py' scripts (just grep for them).
   In particular, look at how 'commands' are created in these scripts,
   and how they are invoked from the outside (e.g. a Makefile).

>> From my side I can tell you that source-to-source transformation is
> difficult process. Py++ introduces two type of trees:
> * declarations tree - something very similar to AST, but one that
> keeps much more information
> * code creators tree - this one keeps objects, which creates code.
> They have to take into account information from declarations tree and
> code creators tree.
> 
> The generated code should meet some standards. In case of Py++ - I
> just cannot afford to generate all code into single file, more over it
> should generate "include" directives only if the code in the file uses
> the functionality.

Understood. Taking this very high-level description, I would express it
like this in Synopsis:

A code-translating processor takes an AST (or similar) as input, and
generates some new AST as output. A formatting processor then traverses
the newly generated AST, 'formatting' it into actual code.

The first processor would need to understand the code mapping, i.e.
understand the boost.python logic, the wrapping, call policies, etc.,
and the second processor would encapsulate formatting constraints, i.e.
how the generated code should be laid out, partitioned into files, etc.

The two processors would be customizable (to differing degrees; for some
aspects a simple parameter is enough, for some you may want to encapsulate
subtasks into classes that can be overridden by users).

> I wrote some document that explains Py++ architecture:
> http://language-binding.net/pyplusplus/documentation/architecture.html
> 
> I hope you find it useful.

I'll read it tonight, thanks !

>> The reason I ask is because I can see yet a different interaction between
>> Synopsis and Py++ (and GCCXML, for that matter):
> 
> I am curious - what do you mean?

* The Synopsis pipeline API is what is exposed to users, who then select
  and customize Py++ code-generating processors, much as they now select
  processors to customize documentation extraction and formatting.

* Py++ merely uses the Synopsis C++ parser, but without using the pipeline
  approach.

* Synopsis uses alternate C++ parsers (such as GCCXML or Elsa) internally,
  and Py++ accesses them through the pipeline API as in the first use case.

>> What if Py++ itself used Synopsis' pipeline language (and thus,
>> scripting frontend),
>> to drive the introspection as well as the code generation ?
> 
> Good C++ parser is critical for s-2-s translators, but this is not enough.
> pygccxml contains type traits - the functionality very similar to
> boost::type_traits.
> It is a must.

I'm not sure what you are saying. Type analysis is part of a real C++ parser.
Are you saying type inspection needs to be accessible through the intermediate
representation(s) ? I fully agree. (And in fact the AST does provide some
information about types, though this is clearly something that needs to be worked
on.)

>> And, as Nathaniel suggests, we could try to hook up gccxml (and may be
>> elsa)
>> as alternative C++ parsers. Obviously, the low level bits will differ
>> a lot
>> (i.e. there is no equivalent of Synopsis's parse tree), but may be for
>> the
>> AST generation it wouldn't make a difference.
> 
> This is the subject I am working right now. I am constructing
> declarations tree ( AST ) from Synopsis and GCC-XML. From my
> experience till now, the trees are very very different. GCCXML
> provides much more information.

Can you elaborate ? I don't doubt what you are saying, I merely
hope that a concrete list of metadata may help us complete the
Synopsis AST. (Since the existing type analysis is rather rudimentary,
most metadata gets stored as strings, e.g. 'premodifiers', 'postmodifiers'),
so it's impossible to do a true query like 'is this a const member ?'

> Also I think it is up to you to decide where Synopsis is going: to be
> high quality C++ parser or thin wrapper around existing ones. (
> Assumption: you will not have time to implement both of them :-) ).

Well, I'm still (somewhat) optimistic that at some point people will emerge
to help hacking on it. Nathaniel is right, this is very much a strategic
question. Also, I don't really see these two as alternatives. Rather, they
are complementary aspects of the same thing.

Regards,
		Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...