Re: Digest Number 459
Eric Bezault <ericb-D6Qt/9opevxWk0Htik3J/[email protected]>
| Newsgroups | gmane.comp.lang.eiffel.gobo.general |
|---|---|
| Organization | Gobo |
| Message-ID | <[email protected]> |
Thomas Beale wrote: > [email protected] wrote: > >>gelex does not support unicode. It only supports 8-bit character >>input stream. However you can consider your unicode files as a >>sequence of bytes (depending on the encoding choosen in the file) >>and use gelex as lexer of byte input stream. This is what is done >>in the XML parser provided in the Gobo Eiffel XML library. >> > > What is the output of this lexer? What do the tokens correspond to > exactly - unicode glyphs? Tokens are element names, attribute names, strings, ... If it were to parse Eiffel classes containing unicode characters, the tokens would be identifiers, keywords, strings, integer values, etc. Exactly the kind of tokens you get with an ASCII parser. -- Eric Bezault mailto:ericb-D6Qt/9opevxWk0Htik3J/[email protected] http://www.gobosoft.com ------------------------ Yahoo! Groups Sponsor --------------------~--> Get to your groups with one click. Know instantly when new email arrives http://us.click.yahoo.com/.7bhrC/MGxNAA/yQLSAA/saFolB/TM --------------------------------------------------------------------~-> To Post a message, send it to: [email protected] To Unsubscribe, send a blank message to: [email protected] Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/gobo-eiffel/ <*> To unsubscribe from this group, send an email to: [email protected] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/