Re: Passing unicode to Email validator

Leandro Lucarella wrote:
> Well, I found the previos "bug" because of this.
> 
> When you pass an unicode string to an Email validator using
> resolve_domain=True, strange things happen (mainly you get "random"
> UnicodeDecodeErrors).
> 
> For example:
> luca@turing:~/sercom:master$ python
> Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from formencode import validators
>>>> v = validators.Email(resolve_domain=True)
>>>> v.to_python(u'[email protected]')
> u'[email protected]'
>>>> v.to_python(u'[email protected]')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.5/site-packages/formencode/api.py", line 371, in to_python 
>     vp(value, state)
>   File "/usr/lib/python2.5/site-packages/formencode/validators.py", line 1233, in validate_python
>     a=DNS.DnsRequest(domain).req().answers
>   File "/var/lib/python-support/python2.5/DNS/Base.py", line 202, in req
>     m.addQuestion(qname, qtype, Class.IN)
>   File "/var/lib/python-support/python2.5/DNS/Lib.py", line 462, in addQuestion
>     self.addname(qname)
>   File "/var/lib/python-support/python2.5/DNS/Lib.py", line 129, in addname
>     self.buf = self.buf + buf
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 0: ordinal not in range(128)
> luca@turing:~/sercom:master$ python
> Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from formencode import validators
>>>> v = validators.Email(resolve_domain=True)
>>>> v.to_python(u'[email protected]')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.5/site-packages/formencode/api.py", line 371, in to_python
>     vp(value, state)
>   File "/usr/lib/python2.5/site-packages/formencode/validators.py", line 1233, in validate_python
>     a=DNS.DnsRequest(domain).req().answers
>   File "/var/lib/python-support/python2.5/DNS/Base.py", line 202, in req
>     m.addQuestion(qname, qtype, Class.IN)
>   File "/var/lib/python-support/python2.5/DNS/Lib.py", line 462, in addQuestion
>     self.addname(qname)
>   File "/var/lib/python-support/python2.5/DNS/Lib.py", line 129, in addname
>     self.buf = self.buf + buf
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xf1 in position 1: ordinal not in range(128)
> luca@turing:~/sercom:master$ python
> Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from formencode import validators
>>>> v = validators.Email(resolve_domain=True)
>>>> v.to_python(u'[email protected]')
> u'[email protected]'
> 
> 
> Please, note the randomness of offending decoded byte (and that sometimes
> it works). I tried to take a look to PyDNS sources and I can't find how
> this "randomness" can happen, but didn't succeded (there are some binary
> manipulations there so it expects its input to be ASCII, because some
> internal buffer is "promoted" to unicode when unicode strings are used and
> that makes all blows).
> 
> Should Email validator first check if the domain is valid ASCII? (at least
> if IDN[1] are not supported by pyDNS).

Yes, that seems reasonable.  I suppose it could also use the punycode 
encoding.  I think that all IDNs are is punycode-encoded unicode, right?

> Should I send a patch? Or you think this is a pyDNS bug?
> 
> [1] http://en.wikipedia.org/wiki/Internationalized_domain_name

Yes, probably pyDNS should handle this more gracefully as well.


-- 
Ian Bicking : [email protected] : http://blog.ianbicking.org

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/