Re: [PATCH] Correctly use cStringIO which only deals in bytestrings.

On Wed, 14 Oct 2009, R. Tyler Ballance wrote:

> I'm uncertain if there's any lossiness in encoding a unicode()
> object 'utf-8' prior to writing to the stream, but this does
> appear to inadvertantly fix another issue we had with mixing UTF-8
> encoded strings and unicode() objects when filling a template.

Worth noting that this patch series has been merged and pushed to
git://github.com/rtyler/cheetah.git/next 

All tests are passing as well, I would really appreciate any testing you
all could spare with this. 

Tar:
	http://github.com/rtyler/cheetah/tarball/cStringIO_alpha
Zip:
	http://github.com/rtyler/cheetah/zipball/cStringIO_alpha


Cheers

> ---
>  cheetah/DummyTransaction.py |    7 +++++--
>  cheetah/Tests/Unicode.py    |    5 ++---
>  2 files changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/cheetah/DummyTransaction.py b/cheetah/DummyTransaction.py
> index 8b30abf..e04a604 100755
> --- a/cheetah/DummyTransaction.py
> +++ b/cheetah/DummyTransaction.py
> @@ -31,8 +31,11 @@ class DummyResponse(object):
>          pass
>          
>      def write(self, value):
> +        if isinstance(value, unicode):
> +            value = value.encode('utf-8')
>          self._outputChunks.write(value)
>  
> +
>      def writeln(self, txt):
>          write(txt)
>          write('\n')
> @@ -43,7 +46,7 @@ class DummyResponse(object):
>              if outputChunks is not None:
>                  return ''.join(outputChunks)
>              else:
> -                return self._outputChunks.getvalue()
> +                return self._outputChunks.getvalue().decode('utf-8')
>          except UnicodeDecodeError, ex:
>              #not sure about the best way to check for non-unicode in StringIO
>              nonunicode = ''
> @@ -93,7 +96,7 @@ class TransformerResponse(DummyResponse):
>              if isinstance(_filter, types.TypeType):
>                  _filter = _filter()
>              return _filter.filter(output)
> -         return output
> +        return output
>  
>  
>  class TransformerTransaction(object):
> diff --git a/cheetah/Tests/Unicode.py b/cheetah/Tests/Unicode.py
> index e4499d9..d627503 100644
> --- a/cheetah/Tests/Unicode.py
> +++ b/cheetah/Tests/Unicode.py
> @@ -168,15 +168,14 @@ class Unicode_in_SearchList_Test(CommandLineTest):
>              'adjective' : u'\u0e22\u0e34\u0e19\u0e14\u0e35\u0e15\u0e49\u0e2d\u0e19\u0e23\u0e31\u0e1a'}])
>          assert template.respond()
>  
> -    def test_ErrorReporting(self):
> +    def test_Thai_utf8(self):
>          utf8 = '\xe0\xb8\xa2\xe0\xb8\xb4\xe0\xb8\x99\xe0\xb8\x94\xe0\xb8\xb5\xe0\xb8\x95\xe0\xb9\x89\xe0\xb8\xad\xe0\xb8\x99\xe0\xb8\xa3\xe0\xb8\xb1\xe0\xb8\x9a'
>  
>          source = '''This is $adjective'''
>          template = self.createAndCompile(source)
>          assert template and issubclass(template, Template)
>          template = template(searchList=[{'adjective' : utf8}])
> -        self.failUnlessRaises(DummyTransaction.DummyResponseFailure, template.respond)
> -
> +        assert template.respond()
>  
>  
>  if __name__ == '__main__':
> -- 
> 1.6.0.2
>

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference

_______________________________________________
Cheetahtemplate-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkrWk4MACgkQFCbH3D9R4W++6QCfVAIZmG1IDdhkSRN1S4zpBdm+
vroAoJPez70P5a+buVhZCX7NuwVG+GSe
=CM0N
-----END PGP SIGNATURE-----