Re: How does python.wy work?

Gracjan Polak <[email protected]> writes:

> I've noticed something interesting in python.wy.
>
> On one hand python.wy in 
>
> indented_block
> : INDENT_BLOCK
> (EXPANDFULL $1 indented_block_body)
> ;
>
> that means that indented_block can only handle INDENT_BLOCK, where
> INDENT_BLOCK must be produced by lexer.
>
> How does the lexer know that it has to produce INDENT_BLOCK instead of
> the usual token stream?
>
> Links:
> https://sourceforge.net/p/cedet/git/ci/master/tree/lisp/cedet/semantic/wisent/python.wy
> https://sourceforge.net/p/cedet/git/ci/master/tree/lisp/cedet/semantic/wisent/python.el

INDENT and DEDENT tokens are generated by analyzing the leading white
space indentation.  The lexer keeps tract of a stack of indentations
encountered so far to push or pop the column numbers.  Whenever new
items are added to this stack, an INDENT token is generated.  For each
item popped, a DEDENT token is generated.

Python specifies exactly what the algorithm ought to be, e.g.,
tab width must be 8.  This algorithm is implemented in elisp.



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140