Re: [PATCH 2/2] No-op the EncodeUnicode filter when it encounters a unicode() object.

On 10/16/2009 06:27 PM, R. Tyler Ballance wrote:
> Since v2.2.0, the handling of unicode objects has been
> gracefully handled by Cheetah, such that the encoding operation
> here is unnecessary (and likely an additional performance hindrance)
> ---
>   cheetah/Filters.py |    2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/cheetah/Filters.py b/cheetah/Filters.py
> index dd65f28..452afc5 100644
> --- a/cheetah/Filters.py
> +++ b/cheetah/Filters.py
> @@ -58,7 +58,7 @@ class EncodeUnicode(Filter):
>           >>>  print t
>           """
>           if isinstance(val, unicode):
> -            return val.encode(encoding)
> +            return val
>           if val is None:
>               return ''
>           return str(val)

I think this is definitely the right thing to do now that all strings 
are unicode objects internally.

This is probably a good time to take a look at the filters in light of 
the recent changes in encoding handling.

The base Filter object allows you to specify an encoding that it'll use 
to encode unicode objects to strs.  Given that every substitution 
variable needs to be converted to unicode before it can be concatenated 
with the rest of the template (which is a unicode object), encoding them 
as strs simply causes them to need to be decoded again in 
DummyTransaction, an unnecessary round-trip.

The base Filter also converts non-basestring objects to strs by default, 
when it would be more efficient to convert them directly to unicode 
objects, and fall back to strs in the failure case.

Both of these changes can be made without changing the API, I'll send a 
patch shortly.  However, the API should probably be changed to reflect 
that it is no longer useful to encode substitution variables to another 
encoding, everything is unicode.  Is this an API change worth making for 
2.4.0?

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference