Re: AW: [viewvc-users] python 2.6 exception when viewing a file containing a

Awesome.  Great to hear it.  Thanks for the update!

On 11/07/2011 08:40 AM, [email protected] wrote:
> Hello Michael,
> 
> Sorry, I forgot. 
> Reason is that in /pygments/lexer.py an exception was not handled right, I did fix it.
> (the exception was text.decode('utf-8') like the normal text.decode, now changed to text.decode('latin1’) )
> 
> So nothing to do directly with ViewVC - and I also saw that the latest version of lexer.py has already this fix.
> 
> So all is cleared,
> 
> Regards
> Tobias
> 
> -----Ursprüngliche Nachricht-----
> Von: C. Michael Pilato [mailto:[email protected]] 
> Gesendet: Montag, 7. November 2011 14:27
> An: Grueninger, Tobias [NETPWR/CHL/ERLG]
> Cc: [email protected]
> Betreff: Re: [viewvc-users] python 2.6 exception when viewing a file containing a
> 
> Tobias, what was the result of your further testing?  (Or has it not yet
> occurred?)
> 
> On 09/15/2011 04:13 AM, Tobias Grüninger wrote:
>>> Thanks for the bug report.  Would it be possible for you to share 
>>> with us the rest of this stack trace?
>>
>> Of course:
>>
>> An Exception Has Occurred
>>
>> Python Traceback
>>
>> Traceback (most recent call last):
>>   File "/usr/lib/viewvc/lib/viewvc.py", line 4317, in main
>>     request.run_viewvc()
>>   File "/usr/lib/viewvc/lib/viewvc.py", line 397, in run_viewvc
>>     self.view_func(self)
>>   File "/usr/lib/viewvc/lib/viewvc.py", line 1763, in view_markup
>>     markup_or_annotate(request, 0)
>>   File "/usr/lib/viewvc/lib/viewvc.py", line 1696, in markup_or_annotate
>>     path[-1], mime_type)
>>   File "/usr/lib/viewvc/lib/viewvc.py", line 1589, in markup_stream_pygments
>>     encoding='utf-8'), ps)
>>   File "/usr/lib/pymodules/python2.6/pygments/__init__.py", line 86, in highlight
>>     return format(lex(code, lexer), formatter, outfile)
>>   File "/usr/lib/pymodules/python2.6/pygments/__init__.py", line 45, in lex
>>     return lexer.get_tokens(code)
>>   File "/usr/lib/pymodules/python2.6/pygments/lexer.py", line 137, in get_tokens
>>     text = text.decode('utf-8')
>>   File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
>>     return codecs.utf_8_decode(input, errors, True)
>> UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 
>> 210: invalid start byte
>>
>> Apparently, it happens in pygments lexer.py where this is not handled correctly:
>>
>> if self.encoding == 'guess':
>>     try:
>>         text = text.decode('utf-8')
>>         if text.startswith(u'\ufeff'):
>>             text = text[len(u'\ufeff'):]
>>     except UnicodeDecodeError:
>>         text = text.decode('utf-8')
>> elif...
>>
>> but should be as 2.6 default is 'strict'
>>
>> if self.encoding == 'guess':
>>     try:
>>         text = text.decode('utf-8')
>>         if text.startswith(u'\ufeff'):
>>             text = text[len(u'\ufeff'):]
>>     except UnicodeDecodeError:
>>         text = text.decode('utf-8', errors='replace') elif...
>>
>> I will test when I can restart my apache.
>>
>> Regards
>> Tobias
>>
>> ------------------------------------------------------
>> http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=4255&dsMessageId=
>> 2839151
>>
>> To unsubscribe from this discussion, e-mail: [[email protected]].
> 
> 
> --
> C. Michael Pilato <[email protected]>
> CollabNet   <>   www.collab.net   <>   Distributed Development On Demand
> 


-- 
C. Michael Pilato <[email protected]>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

------------------------------------------------------
http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=4255&dsMessageId=2876017

To unsubscribe from this discussion, e-mail: [[email protected]].

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk639nsACgkQokEGqRcG/W5sDwCgx7tD00E+kqQTHVAZQzYjd/O1
OhYAoI53kh78WJ0T3ahcGSniU0z8G5cY
=3JiH
-----END PGP SIGNATURE-----