AW: [viewvc-users] python 2.6 exception when viewing a file containing a

Hello Michael,

Sorry, I forgot. 
Reason is that in /pygments/lexer.py an exception was not handled right, I did fix it.
(the exception was text.decode('utf-8') like the normal text.decode, now changed to text.decode('latin1’) )

So nothing to do directly with ViewVC - and I also saw that the latest version of lexer.py has already this fix.

So all is cleared,

Regards
Tobias

-----Ursprüngliche Nachricht-----
Von: C. Michael Pilato [mailto:[email protected]] 
Gesendet: Montag, 7. November 2011 14:27
An: Grueninger, Tobias [NETPWR/CHL/ERLG]
Cc: [email protected]
Betreff: Re: [viewvc-users] python 2.6 exception when viewing a file containing a

Tobias, what was the result of your further testing?  (Or has it not yet
occurred?)

On 09/15/2011 04:13 AM, Tobias Grüninger wrote:
>> Thanks for the bug report.  Would it be possible for you to share 
>> with us the rest of this stack trace?
> 
> Of course:
> 
> An Exception Has Occurred
> 
> Python Traceback
> 
> Traceback (most recent call last):
>   File "/usr/lib/viewvc/lib/viewvc.py", line 4317, in main
>     request.run_viewvc()
>   File "/usr/lib/viewvc/lib/viewvc.py", line 397, in run_viewvc
>     self.view_func(self)
>   File "/usr/lib/viewvc/lib/viewvc.py", line 1763, in view_markup
>     markup_or_annotate(request, 0)
>   File "/usr/lib/viewvc/lib/viewvc.py", line 1696, in markup_or_annotate
>     path[-1], mime_type)
>   File "/usr/lib/viewvc/lib/viewvc.py", line 1589, in markup_stream_pygments
>     encoding='utf-8'), ps)
>   File "/usr/lib/pymodules/python2.6/pygments/__init__.py", line 86, in highlight
>     return format(lex(code, lexer), formatter, outfile)
>   File "/usr/lib/pymodules/python2.6/pygments/__init__.py", line 45, in lex
>     return lexer.get_tokens(code)
>   File "/usr/lib/pymodules/python2.6/pygments/lexer.py", line 137, in get_tokens
>     text = text.decode('utf-8')
>   File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
>     return codecs.utf_8_decode(input, errors, True)
> UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 
> 210: invalid start byte
> 
> Apparently, it happens in pygments lexer.py where this is not handled correctly:
> 
> if self.encoding == 'guess':
>     try:
>         text = text.decode('utf-8')
>         if text.startswith(u'\ufeff'):
>             text = text[len(u'\ufeff'):]
>     except UnicodeDecodeError:
>         text = text.decode('utf-8')
> elif...
> 
> but should be as 2.6 default is 'strict'
> 
> if self.encoding == 'guess':
>     try:
>         text = text.decode('utf-8')
>         if text.startswith(u'\ufeff'):
>             text = text[len(u'\ufeff'):]
>     except UnicodeDecodeError:
>         text = text.decode('utf-8', errors='replace') elif...
> 
> I will test when I can restart my apache.
> 
> Regards
> Tobias
> 
> ------------------------------------------------------
> http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=4255&dsMessageId=
> 2839151
> 
> To unsubscribe from this discussion, e-mail: [[email protected]].


--
C. Michael Pilato <[email protected]>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

------------------------------------------------------
http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=4255&dsMessageId=2875999

To unsubscribe from this discussion, e-mail: [[email protected]].