Re: PEP 263 and Japanese native encodings
"M.-A. Lemburg" <[email protected]>
| Newsgroups | gmane.comp.python.internationalization |
|---|---|
| Organization | eGenix.com Software GmbH |
| Message-ID | <[email protected]> |
"Martin v. Loewis" wrote: > > "M.-A. Lemburg" <[email protected]> writes: > > > Martin's patch leaves these "minor" issues to the tokenizer > > and that's good :-) > > > > I only wanted to give a very simple > > example of what the original idea was when I added "ASCII > > compatible encoding" to the PEP -- basically to simplify > > the coding parsing part. > > In my implementation, the "ASCII superset" restriction is stronger, > though: the tokenizer needs to find the end of a string without > decoding it. That is not possible for some of the encodings that pass > your "ASCII superset" test. I hope I have clarified the intent of the definition that was in the PEP -- I've removed it a few days ago and replaced it with a hopefully better description of what was meant. Your implementation is just fine in this respect (at least from looking at it ;-). Are there any other common encodings which could have trouble with the implementation besides two or more byte encodings ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/