Moving along...

Hi!
	I've been late joining the mail list, and I feel a little guilty
since I put the bug in Andrew's ear to champion the SIG.
	The first item on my agenda is talking about PEP 269.  I think
Guido may have retired it already, but I was hoping someone here might
like to talk through the modifications needed to accomplish the following:
	1. Implement PEP 269, as written or slightly modified for a more
sane API.
	2. Possibly supersede PEP 269, by expanding its scope to allow
plugable lexors.
	I know Samuele does not want pgen to be exposed as a part of
CPython, but I am prone to do it because it is easy (actually, I ran into
a serious memory collection problem, and it caused me to shelve my work
until I had more time and drive to mess with it.)
	I do think Guido would still be willing to consider a pgen API,
and I think that maybe a Java port of the pgen C code might suppress any
Jython flak.  I personally think Jim H's choice of ANTLR was a bit too
Java-esque as opposed to Pythonic.  IMHO, Jim was only looking for a quick
solution as opposed to what may have been the right solution in the long
run.  I think the Jython parser should be derived from the same grammar
file Python uses.  This would ease the catch up game the Jython folks have
to play for every grammar change made to the Python language
specification (such as optional static typing, *smirk*).
	In the large (more within the scope of the SIG), I have some notes
for my Basil project that may extend out to our SIG.  These are at:

http://jonathanriehl.net/cgi-bin/moin/BasilParserIntegrations

	Looking over my notes, I have it steeped in my own lame
Basil-speak, and perhaps can boil what I am talking about down to the
following requirements:
	* The parser generator library should provide a set of classes to
represent an input grammar.  The set of classes should form an
intermediate language that is sent to a parser generation back end.
	* The parser generator library should at least provide one input
language (EBNF) that translates an input file into the parser generator
intermediate langauge.
	* The parser generator library should provide one parser
generator back end.
	* The parser generator library should provide or resolve upon the
use of one generic AST (abstract syntax tree) data type.  (Serveral AST
data types are already available in other modules, IIRC.)
	* Parser generator back ends that are part of the parser
generator library should at least provide a default output of
Python code.  The generated Python parser should parse input
tokens into instances of the AST data type.
	* The parser generator library may provide a set of
transformations for the parser generator intermediate language.  Such
transformations could simplify the grammar, detect and/or remove
left/right recursion, etc.
	* The parser generator library may provide one or more code
generators that translate the parser generator intermediate language back
into a parser generator input grammar.  Such a system would allow
translation of input grammars, such as translating Bison input files to
pgen format, should multiple input languages be supported.
	For you requirements engineers out there: the shoulds above may
become shalls, or MUSTs in IETF speak, should they be accepted by the SIG.
Accepted mays should become shoulds, which could be interpreted as IETF
SHOULD's or MAY's, depending on which way the wind is blowing that day.
*smirk*
	Data flow of such a system may, might, could, or possibly should,
would the SIG so agree, look like this:

Input grammar -> Input grammar parser => Parser generator IL

Parser generator IL -> Parser generator back end => Parser for input
grammar

Input language -> Parser for input grammar => AST instance

(Parser generator IL -> Reverse input grammar translator => Input grammar)

	Where one or more input grammar parsers are supported by the
library, with optional reverse engineering; and one or more parser
generator routines are supported by the system.
	Issues:
	What about attributed grammars (I am not sure I am using the right
terminology here; I am talking about the union data type tree nodes used
in Bison reductions/actions)?  Authors familiar with YACC/Bison are going
to want to couple semantic actions with reductions in the input grammar.
Should we let them do this or not?  Might we throw out the actions (as I
do in my Basil Bison front end) or should the IL still support these? In
some cases this is required for interaction with the lexor (such as the
typedef/identifier token resolution in the C++ language).
	In other cases, LL(n) parsers use heuristics to guide reductions
(rather than simply following each possible reduction and going with the
first one that doesn't throw an exception).  Do we want to provide support
for these heuristics in the intermediate language?  Do we want to provide
support for these in one or more input front ends?
	I personally don't like coupling semantics with grammar because it
makes both of them less readable, and constrains reuse of the grammar.
For me the ideal example of this is the GNU C++ grammar.  Some of us want
to write software engineering tools, not go straight to the GCC
intermediate language.  Perhaps there could be some means of inlining
actions that would be taken by a tree walker generator.  Tree walkers for
grammars could be automatically generated to walk the default AST output
of the generated parser.
	What should the output of a Python parser generator be?  Python
code?  A parser object that could be serialized to multiple formats?
(Such as a state machine that could be serialized as Python code, C/C++
code, Java, or directly used as input to a Python parser library.)

Based on list traffic, I hope to get everyone's input by early 2004.
Till then: cheers!
-Jon