Re: [patch] UnicodeDecodeError in tools.Replacer.replace with non-ascii characters

Hrm...  I think the better fix is to change _VAR_REGEXP from unicode to 
a regular string.  One of the efforts for PyBlosxom 1.5 is to make the 
lifecycle use utf-8 encoded strings across the board.

If you take the u out of the _VAR_REGEXP, does that fix the issue, too?


Andrey Bondarenko wrote:
> Hi,
> 
> I have troubles with pyblosxom 1.5-r1330 deployed in Apache 2.2 with
> ModWSGI. I’m writing entries in utf-8 with rst plugin. Config.py
> contains py["blog_encoding"] = "utf-8". But when I’m trying to access
> my blog I get the error:
> 
> mod_wsgi (pid=504): Exception occurred processing WSGI script 
> '.../apps/blog-ups/pyblosxom.wsgi'.
>  Traceback (most recent call last):
>    File "...\\Pyblosxom\\pyblosxom.py", line 446, in __call__
>      return [self.run_pyblosxom(env, start_response)]
>    File "...\\Pyblosxom\\pyblosxom.py", line 438, in run_pyblosxom
>      p.run()
>    File "...\\Pyblosxom\\pyblosxom.py", line 163, in run
>      blosxom_handler(self._request)
>    File "...\\Pyblosxom\\pyblosxom.py", line 916, in blosxom_handler
>      renderer.render()
>    File "...\\Pyblosxom\\renderers\\blosxom.py", line 313, in render
>      content = self.renderContent(self._content)
>    File "...\\Pyblosxom\\renderers\\blosxom.py", line 257, in renderContent
>      outputbuffer.append(self.renderTemplate(vars, "story", override=1))
>    File "...\\Pyblosxom\\renderers\\blosxom.py", line 361, in renderTemplate
>      finaltext = tools.parse(self._request, entry, template)
>    File "...\\Pyblosxom\\tools.py", line 458, in parse
>      return _VAR_REGEXP.sub(replacer.replace, template)
>  UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: 
> ordinal not in range(128)
> 
> I think, the cause of error is forced conversion to string in
> tools.Replacer.parse.  _VAR_REGEXP is Unicode so, sub() have to
> convert values returned by tools.Replacer.parse back from string to
> Unicode. Because my data is in utf-8 default acsii codec can’t do the
> conversion.
> 
> I was able to get pyblosxom working with the patch:
> 
> Index: Pyblosxom/tools.py
> ===================================================================
> --- Pyblosxom/tools.py                (revision 1330)
> +++ Pyblosxom/tools.py             (working copy)
> @@ -429,12 +429,9 @@
>                  # functions took no arguments
>                  r = r()
>  
> -        # convert non-strings to strings
> -        if not isinstance(r, str):
> -            if isinstance(r, unicode):
> -                r = r.encode(self._encoding)
> -            else:
> -                r = str(r)
> +        # convert strings to unicode
> +        if isinstance(r, str):
> +            r = unicode(r, self._encoding)
>  
>          return r
> 
> 
> Best regards,
> Andrey V. Bondarenko
> 
> 
> ------------------------------------------------------------------------------
> Enter the BlackBerry Developer Challenge  
> This is your chance to win up to $100,000 in prizes! For a limited time, 
> vendors submitting new applications to BlackBerry App World(TM) will have 
> the opportunity to enter the BlackBerry Developer Challenge. See full prize 
> details at: http://p.sf.net/sfu/blackberry
> _______________________________________________
> Pyblosxom-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/pyblosxom-devel

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have 
the opportunity to enter the BlackBerry Developer Challenge. See full prize 
details at: http://p.sf.net/sfu/blackberry
_______________________________________________
Pyblosxom-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-devel