Re: Prothon's parser is too naive

On Wed, 07 Jul 2004 22:35:32 -0700 Paul Prescod <[email protected]> wrote:

> I predict that Prothon's parser will evolve to be more and more complex 
> over time and it is best to bite the bullet and put in place a more 
> advanced architecture right now. I was going to write a treatise on why 
> this is so but I'm hoping that I can just point to a presentation I did 
> for the Vancouver Python User's Group (any Vancouverites lurking?). It 
> describes why Python's two-pass compiler is actually too simple and is 
> thus being refactored in the so-called AST branch.
> 
> The gist is that Prothon's architecture looks like this:
> 
> parser -> compiler
> 
> Python's looks like this:
> 
> parser -> parse tree -> compiler
> 
> Python's is evolving "someday" to look like this:
> 
> parser -> parse tree -> AST -> compiler
> 
> The AST is helpful for a few reasons.
> 
>   1. It really simplifies the implementation of the compiler. Parsers 
> are complex and they change implementation when the grammar changes. 
> compilers are complex and change for optimizations. Tieing them together 
> is bad.
> 
>   2. Without the AST, every change to the grammar requires a change to 
> the compiler. Again this limits the amount of effort any sane person 
> will want to put into understanding the grammar.
>
I thought this was deliberate to discourage petty changes to the language. :-)
A feature must be truly wanted before anyone is willing to muck with the
parser/compiler.
 
>   3. The AST is useful for third-party programs like Prothon to X 
> compilers and Prothon IDEs. In Python, ASTs are already being used this 
> way even though they are implemented in pure Python and they are 
> inefficient as hell. Once again it is a big bonus that these tools are 
> isolated from changes to the grammar.
>
You are of course refering to the Python compiler package as opposed to the
parser module. The documentation refers to the trees both modules produce
as ASTs. But the compiler.ast trees are certainly more abstract than the
parser "AST", which is the Python parse tree.
 
>   4. Many optimizations become obvious in the AST. In the compiler you 
> have a harder time having a global view of if, e.g. a variable is ever 
> used later. Pysco and Java translate from the bytecode but I think that 
> there is a case to be made that the AST is better...
> 
Remember that Prothon has no equivalent of Python's compiled files. Modules
are compiled every time a program is run. Unless that changes one must
consider compiler performance. And given the dynamic nature of prototypes
optimizations that work in a typed language will not apply to Prothon.

Lenard Lindstrom
<[email protected]>