various gelex, unicode

Will answer a few posts here as I am on digest....

Message 4 From: "Eric Bezault" ericb-D6Qt/9opevxWk0Htik3J/[email protected] Date: Tue May 9, 2006 
7:38pm(PDT) Subject: Re: Digest Number 459 >

 >>
 >>    
 > > What is the output of this lexer? What do the tokens correspond to
 > > exactly - unicode glyphs?
 >  

Tokens are element names, attribute names, strings, ...

If it were to parse Eiffel classes containing unicode
characters, the tokens would be identifiers, keywords,
strings, integer values, etc. Exactly the kind of tokens
you get with an ASCII parser.

TB> does it make it easier to use normal gelex/yacc classes if the 
unicode characters are restricted to being inside String expressions, 
i.e. inside "" markers?

Message 6 From: "Eric Bezault" ericb-D6Qt/9opevxWk0Htik3J/[email protected] Date: Wed May 10, 2006 
3:52am(PDT) Subject: Re: gelex Q Thomas Beale wrote:

 > > In our gelex/geyacc archetype parser I have a state during which 
text in
 > > a sub-syntax is consumed. The lexer is in that state when the EOF is
 > > reached. The problem is that when EOF is reached, two things need 
to be
 > > done:
 > > * set last_token to a meaningful value, and put the text scanned so 
far
 > > into last_string
 > > * call terminate, which I think sets last_token to 0
 > >
 > > But I can't do these both in the same fragment of code.
 > >
 > > I tried doing it like this:
 > >
 > > <ss><<EOF>> {
 > >        -- unread_character(text.item(1))  -- would this work?
 >  

....

<SS>{
    <<EOF>> {
        print ("Scanner T_SS%N")
        last_token := T_SS
        last_string_value := "Hello"
        set_start_condition (INITIAL)
    }
    (.|\n) {
        -- Ignore unmatched characters
    }
}

<<EOF>> {
    print ("Scanner EOF%N")
    terminate
}

(.|\n) {
    -- Ignore unmatched characters
}


TB> This is the bit that works for me - I didn't realise you could nest 
patterns and states like that in the .l file...


All this stuff about default rule is explained in the doc:

    http://www.gobosoft.com/eiffel/gobo/gelex/matching_rules.html

TB> I did read that pretty carefully, but it doesn't explain the ability 
to nest patterns...but anyway, I am not complaining, these tools are 
brilliant and I am always happy that they cause me so little 
trouble...it's interesting to see how hard it is for our Java and C# 
programmers to do the same things - the parsing tools in those languages 
are quite poor.

thanks Eric - problem is fixed now... - thomas


-- 
___________________________________________________________________________________
CTO Ocean Informatics (http://www.OceanInformatics.biz)
Research Fellow, University College London (http://www.chime.ucl.ac.uk)
Chair Architectural Review Board, openEHR (http://www.openEHR.org)




------------------------ Yahoo! Groups Sponsor --------------------~--> 
Home is just a click away.  Make Yahoo! your home page now.
http://us.click.yahoo.com/DHchtC/3FxNAA/yQLSAA/saFolB/TM
--------------------------------------------------------------------~-> 

To Post a message, send it to:   [email protected]
To Unsubscribe, send a blank message to: gobo-eiffel-unsubscribe-B11MqFFcr06IwRZHo2/[email protected] 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/gobo-eiffel/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/