[issue2551008] Wrong header encoding handling in mailgw.py

New submission from Ezio Melotti:

RoundupMessage._decode_header (formerly known as
Message._decode_header_to_utf8) is incorrect when the encoding is missing:

    def _decode_header(self, hdr):
        parts = []
        for part, encoding in decode_header(hdr):
            if encoding:
                part = part.decode(encoding)
            parts.append(part)
        return ''.join([u2s(p) for p in parts])

If the encoding is specified, the parts will be decoded to a list of
unicode strings, if it isn't, parts will be a list of byte strings.  In
the latter case, u2s() will fail to encode the byte strings on Python 2
if they contain non-ascii characters, and it will always fail on Python
3 since byte strings don't have an .encode() method.

I fixed this downstream by attempting the decoding using utf-8 first and
falling back on iso-8859-1 if that fails:
* https://hg.python.org/tracker/roundup/rev/d7454b42b914
* http://psf.upfronthosting.co.za/roundup/meta/issue668

The code on 1.5 is slightly different, but the logic is the same.

----------
components: Mail interface
keywords: Effort-Low
messages: 6276
nosy: ezio.melotti
priority: normal
severity: normal
status: new
title: Wrong header encoding handling in mailgw.py
type: behavior
versions: 1.5, 1.6, devel

________________________________________________
Roundup tracker <[email protected]>
<http://issues.roundup-tracker.org/issue2551008>
________________________________________________