Prothon's parser is too naive

I predict that Prothon's parser will evolve to be more and more complex 
over time and it is best to bite the bullet and put in place a more 
advanced architecture right now. I was going to write a treatise on why 
this is so but I'm hoping that I can just point to a presentation I did 
for the Vancouver Python User's Group (any Vancouverites lurking?). It 
describes why Python's two-pass compiler is actually too simple and is 
thus being refactored in the so-called AST branch.

The gist is that Prothon's architecture looks like this:

parser -> compiler

Python's looks like this:

parser -> parse tree -> compiler

Python's is evolving "someday" to look like this:

parser -> parse tree -> AST -> compiler

The AST is helpful for a few reasons.

  1. It really simplifies the implementation of the compiler. Parsers 
are complex and they change implementation when the grammar changes. 
compilers are complex and change for optimizations. Tieing them together 
is bad.

  2. Without the AST, every change to the grammar requires a change to 
the compiler. Again this limits the amount of effort any sane person 
will want to put into understanding the grammar.

  3. The AST is useful for third-party programs like Prothon to X 
compilers and Prothon IDEs. In Python, ASTs are already being used this 
way even though they are implemented in pure Python and they are 
inefficient as hell. Once again it is a big bonus that these tools are 
isolated from changes to the grammar.

  4. Many optimizations become obvious in the AST. In the compiler you 
have a harder time having a global view of if, e.g. a variable is ever 
used later. Pysco and Java translate from the bytecode but I think that 
there is a case to be made that the AST is better...

http://www.google.com/search?q=ast+parse+tree+intermediate&ie=UTF-8&oe=UTF-8

  Paul Prescod