Re: source-to-source translators

On 6/16/05, Stefan Seefeld <[email protected]> wrote:
> Fine. I hope you don't mind me taking the discussion to the synopsis mailing.
...Subscribing, done :-)

> > I'd like to say that adding an other back end to pygccxml is something
> > that I thought about this. I tried to do it using Microsoft Visual
> > Studio API, but failed. It was too buggy.
> > Here is my reasons to add sysnopsis as back end to pygccxml:
> > 1. performance improvements
> > 2. development processes integration: creating code and documentation
> > for the code could be done from the same parse tree.
> > ( I think that is what you meant under " 'processor pipeline", right? )
> 
> In synopsis, pipelines are a compositional pattern by which processors
> are chained into composite processors
> (http://synopsis.fresco.org/docs/Tutorial/pipeline.html). I have found
> this idiom to be very productive, and while it is currently only directly
> applied to ASTs, I'm hoping that it can be applied to other representations
> as well. Most importantly to the user, pipelines are very flexible and
> extensible. I came up with the current design specifically to solve the
> problem I had when trying to express *lots* of parameters that should
> apply to the data processing, some related to the parsing itself, others
> to the formatting.

I know this design pattern. What about performance? Did you see change
for the worse?

> [...]
> 
> >>As far as I understand, for source-to-source translation you need a
> >>correct parse tree, but you don't need all aspects of type analysis.
> >>Am I wrong ?
> >
> >
> > I think I understand what you mean under "correct type analysis", but
> > I am not sure.
> > Lets try to see small example from "real life":
> >
> > some_type session_t::operator()(boost::call_traits<packet_t>::param_type
> > some_type){
> > ...
> > }
> >
> > And I add constraint that exists in boost.python library:
> > functions that takes as argument pointer to function could not be exported.
> >
> > Now if I compile this code using GCCXML I can find out
> > "boost::call_traits<packet_t>::param_type" expression type. If the
> > expression type
> > is pointer to function I can: skip generation of the function or to
> > generate it within
> > comments with reference to manuals. This is what you meant under
> > "correct type analysis", right? Call policies is an other area where
> > user need type analysis.
> 
> Indeed. The Synopsis C++ parser right now creates a parse tree and a symbol table.
> When looking up 'session_t::operator()' it would find *all* overloaded call operators
> for 'session_t'. What is missing is overload resolution, as well as (partial)
> template specialization, as both require complete type analysis, and until that
> is implemented you won't know *which* call operator the above snippet is referring to.
> 
> Synopsis at present does some form of type analysis when translating the parse
> tree to an AST, so it *might* already work.

Do you have list of cases when synopsis type analysis will not work? 
I'd like to see it before integrating pygccxml with synopsis.

> > Now about source-to-source translator:
> >
> > You said about this but didn't explained you self. Here is my point of
> > view to the subject:
> > Source-to-source translator could be very complex and adding some
> > domain specific language can do the work.
> 
> Right.
> 
> > One of the famous examples
> > is XML and XSLT. If we do some analogy then we have parse tree as XML
> > and some unspecified domain specific language as XSLT. From one thing
> > we gain a lot of power, but we insert a lot of complexity. May it
> > worse it I don't know.
> 
> I don't think that XML/XSLT is a sensible choice for a domain-specific
> language here, notably because it isn't specific to the domain of source
> transformation at all. :-)

I didn't try to set XML\XSLT as domain-specific language. It was just
an example.
I actually propose AST \ ASTT ( transformation ).

> > An other example from the "real life".
> > boost.python library has constraint: every based class should be
> > exported before it's derives. Topological sort of class declarations -
> > it is very simple to implement using high level general purpose
> > language. It could be very difficult to implement using domain
> > specific language.
> 
> I'm not sure I understand what you mean by 'domain specific language'.
> If it is really specific to the domain, it makes it *easy* to express
> semantic constraints that arise from the domain itself, as the language
> is designed specifically for that purpose.
> 
> But that nitpicking aside, I believe I understand what you are saying:
> To solve the problem of 'source translation' in general, we need a
> language that is expressive enough for this purpose, i.e. that lets
> us express constraints, conditions, as well as actual transformations.

Yes, this is exactly what I mean. This solution is very powerful, but it has 
it's own problems. One of them is complexity. For example - boost.spirit
library introduces domain specific language, using C++ syntax. We could
build something similar ( in idea ) using python. But I am not sure that
users will learn an other language to complete the task ( exposing C++
declarations
to python - that is the main case for me ). I am not talking about the
challenge
to build such language. On other problem is "time to market". It will
take a huge amount
of time to build such thing. 

> > May be I miss something or did not get your ideas right. May be I
> > completely wrong.
> > But, I thing that result of this discussion could be simple and
> > powerful code generation tool.
> 
> Yes ! I think this has to be an iterative process. In the end, I'd like
> to see a domain-specific language that operates mostly with declarations
> (constraint -> transformation), but before condensing the language I'm sure
> some conventional imperative steps are required. Therefor, I think that
> python is an ideal playground.

You are right. 

> Regards,
>                 Stefan
> 

Right now I am writing documentation and examples for my project ( py++ ).
When it will be ready, if you don't mind, I will contact you again, in
order we can
transform, :), our discussion to something useful. 

Thanks, 

Roman