Re: source-to-source translators

Roman Yakovenko wrote:
> Hi. You drop some nice ideas on boost.python list.
> If you don't mind I would like to discuss those ideas with you.
> I think this subject has nothing to do with boost.python so I took it
> from the list.

Fine. I hope you don't mind me taking the discussion to the synopsis mailing.

> I'd like to say that adding an other back end to pygccxml is something
> that I thought about this. I tried to do it using Microsoft Visual
> Studio API, but failed. It was too buggy.
> Here is my reasons to add sysnopsis as back end to pygccxml:
> 1. performance improvements
> 2. development processes integration: creating code and documentation
> for the code could be done from the same parse tree.
> ( I think that is what you meant under " 'processor pipeline", right? )

In synopsis, pipelines are a compositional pattern by which processors
are chained into composite processors
(http://synopsis.fresco.org/docs/Tutorial/pipeline.html). I have found
this idiom to be very productive, and while it is currently only directly
applied to ASTs, I'm hoping that it can be applied to other representations
as well. Most importantly to the user, pipelines are very flexible and
extensible. I came up with the current design specifically to solve the
problem I had when trying to express *lots* of parameters that should
apply to the data processing, some related to the parsing itself, others
to the formatting.

[...]

>>As far as I understand, for source-to-source translation you need a
>>correct parse tree, but you don't need all aspects of type analysis.
>>Am I wrong ?
> 
> 
> I think I understand what you mean under "correct type analysis", but
> I am not sure.
> Lets try to see small example from "real life":
> 
> some_type session_t::operator()(boost::call_traits<packet_t>::param_type
> some_type){
> ...
> }
> 
> And I add constraint that exists in boost.python library:
> functions that takes as argument pointer to function could not be exported.
> 
> Now if I compile this code using GCCXML I can find out 
> "boost::call_traits<packet_t>::param_type" expression type. If the
> expression type
> is pointer to function I can: skip generation of the function or to
> generate it within
> comments with reference to manuals. This is what you meant under
> "correct type analysis", right? Call policies is an other area where
> user need type analysis.

Indeed. The Synopsis C++ parser right now creates a parse tree and a symbol table.
When looking up 'session_t::operator()' it would find *all* overloaded call operators
for 'session_t'. What is missing is overload resolution, as well as (partial)
template specialization, as both require complete type analysis, and until that
is implemented you won't know *which* call operator the above snippet is referring to.

Synopsis at present does some form of type analysis when translating the parse
tree to an AST, so it *might* already work.

> Now about source-to-source translator:
> 
> You said about this but didn't explained you self. Here is my point of
> view to the subject:
> Source-to-source translator could be very complex and adding some
> domain specific language can do the work.

Right.

> One of the famous examples
> is XML and XSLT. If we do some analogy then we have parse tree as XML
> and some unspecified domain specific language as XSLT. From one thing
> we gain a lot of power, but we insert a lot of complexity. May it
> worse it I don't know.

I don't think that XML/XSLT is a sensible choice for a domain-specific
language here, notably because it isn't specific to the domain of source
transformation at all. :-)

> An other example from the "real life".
> boost.python library has constraint: every based class should be
> exported before it's derives. Topological sort of class declarations -
> it is very simple to implement using high level general purpose
> language. It could be very difficult to implement using domain
> specific language.

I'm not sure I understand what you mean by 'domain specific language'.
If it is really specific to the domain, it makes it *easy* to express
semantic constraints that arise from the domain itself, as the language
is designed specifically for that purpose.

But that nitpicking aside, I believe I understand what you are saying:
To solve the problem of 'source translation' in general, we need a
language that is expressive enough for this purpose, i.e. that lets
us express constraints, conditions, as well as actual transformations.

> May be I miss something or did not get your ideas right. May be I
> completely wrong.
> But, I thing that result of this discussion could be simple and
> powerful code generation tool.

Yes ! I think this has to be an iterative process. In the end, I'd like
to see a domain-specific language that operates mostly with declarations
(constraint -> transformation), but before condensing the language I'm sure
some conventional imperative steps are required. Therefor, I think that
python is an ideal playground.

Regards,
		Stefan