getmail failing for messages with invalid uncode

Michael Grant <[email protected]> Wed, 13 Oct 2021 06:06:05 -0400
Newsgroups	gmane.mail.getmail.user
Message-ID	<[email protected]>
I'm using getmail to suck down my spam folder and further process it.
If a spam message contains an invalid unicode sequence, I get the
following error:

    Exception: please read docs/BUGS and include the following information in any bug report:

    getmail version 6.14
    Python version 3.9.2 (default, Feb 28 2021, 17:03:44)
    [GCC 10.2.1 20210110]

    Unhandled exception follows:
    File "/usr/bin/getmail", line 932, in main
    success = go(configs, options.idle)
    File "/usr/bin/getmail", line 244, in go
    msg = mail_filter.filter_message(msg, retriever)
    File "/usr/lib/python3/dist-packages/getmailcore/filters.py", line 79, in filter_message
    exitcode, newmsg, err = self._filter_message(msg)
    File "/usr/lib/python3/dist-packages/getmailcore/filters.py", line 289, in _filter_message
    msg.add_header('X-getmail-filter-classifier', line)
    File "/usr/lib/python3/dist-packages/getmailcore/message.py", line 210, in add_header
    self.__msg[name] = Header(content.rstrip(), 'utf-8')
    File "/usr/lib/python3.9/email/header.py", line 217, in __init__
    self.append(s, charset, errors)
    File "/usr/lib/python3.9/email/header.py", line 296, in append
    s = s.decode(input_charset, errors)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 279: invalid start byte

    Please also include configuration information from running getmail
    with your normal options plus "--dump".
    bash: eval: -g: invalid option
    eval: usage: eval [arg ...]

Here is the offending line in the email:

b'<B>Kindly get back to me for directives on how to receive your fund without any delay, hence we have agreed with DEUTSCHE Bank , When I received your respond I shall send you a copy of your Funds Approval Certificate from the United Nations. Please send your complete information\x92s, Full Name&nbsp; Address .........Telephone number&nbsp; ..............</B></DIV>'

The 0x92 character is illegal utf-8 in this sequence, it can only be a
continuation byte.  Other illegal utf-8 encodings also set off this error.

This doesn't look like it's part of getmail.  Looks like it's part of
libpython3.9-minimal:

    % dpkg -S "/usr/lib/python3.9/email/header.py"
    libpython3.9-minimal:amd64: /usr/lib/python3.9/email/header.py

I've already opened a bug report on this and sent who appears to be
the author a message but it doesn't seem like anyone is actively
wanting to fix this.

Just wondering if anyone has a decent work-around?

Michael Grant
signature.asc (application/pgp-signature, 833 B)