Re: First strawman for UTF-8 headers proposal

Hello Keith,


At 20:35 03/11/30 -0500, Keith Moore wrote:

>One simple example.  Bernstein and others have pointed out that it's 
>easier to parse header fields with address lists from the right to the 
>left rather than from the left to the right, because this requires less 
>lookahead.  It's still possible to do this with UTF-8 (particularly if you 
>do lexical analysis left-to-right and parsing right-to-left), but it's 
>probably not a trivial change to existing code.

Can you give more details? As long as lexing or parsing treats anything
non-ascii the same, things shouldn't change at all (as long as the code
is 8-bit clean). If different non-ASCII characters have to lex or parse
differently, then you have to use tables, do some conversion, or do some
hand-coding with a byte-by-byte approach, and the complexity of this is
virtually the same whether you go one way or the other. If you already
have the UTF-8 forward code, then that's not trivial to change to
reverse scanning code. But if you only have ASCII, the changes to move
to UTF-8 are about the same for both directions, except that you
probably have a bigger chance to find already existing code that
goes forward.


Regards,    Martin.