[issue2551209] mailgw submission fails, nosy mail gets sent, message not added to database

Marcus Priesch <[email protected]> Tue, 07 Jun 2022 16:20:11 +0000

Newsgroups	gmane.comp.bug-tracking.roundup.devel
Message-ID	<[email protected]>

New submission from Marcus Priesch:

as requested per issue2551203, here are the requested email messages

the problematic characters dropped into the mail by copy/paste from the devtools browser window.

however, maybe the header line:

    Content-Type: text/plain; charset=UTF-8; format=flowed

should have triggered conversion to utf-8 instead of ascii.

when opening message_to_mailgw.txt in python with ascii codec, you get exactly the same error message:

>>> open ("message_to_mailgw.txt","r",encoding="ascii").read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python-exec/python3.9/../../../lib/python3.9/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 1817: ordinal not in range(128)


thanks,
marcus.

----------
components: Database, Mail interface
files: failed_submission_email.txt, message_to_mailgw.txt
messages: 7563
nosy: marcus.priesch
severity: normal
status: new
title: mailgw submission fails, nosy mail gets sent, message not added to database
type: crash
versions: 1.6.1

_________________________________________________
Roundup tracker <[email protected]>
<https://issues.roundup-tracker.org/issue2551209>
_________________________________________________

_______________________________________________
Roundup-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/roundup-devel

failed_submission_email.txt (text/plain, 9.3 KB)

Sender: [email protected]
Content-Type: multipart/mixed; boundary="===============0112997174240416991=="
MIME-Version: 1.0
Subject: Failed issue tracker submission
To: [email protected]
From: Roundup tracker <[email protected]>
Date: Tue, 07 Jun 2022 09:56:18 +0000
Precedence: bulk
Message-Id: <[email protected]>
Content-Length: 7104

--===============0112997174240416991==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit



There was a problem with the message you sent:
   'ascii' codec can't decode byte 0xef in position 1451: ordinal not in range(128)



Mail Gateway Help
=================
Incoming messages are examined for multiple parts:
 . In a multipart/mixed message or part, each subpart is extracted and
   examined. The text/plain subparts are assembled to form the textual
   body of the message, to be stored in the file associated with a "msg"
   class node. Any parts of other types are each stored in separate files
   and given "file" class nodes that are linked to the "msg" node.
 . In a multipart/alternative message or part, we look for a text/plain
   subpart and ignore the other parts.
 . A message/rfc822 is treated similar tomultipart/mixed (except for
   special handling of the first text part) if unpack_rfc822 is set in
   the mailgw config section.

Addresses
---------
All of the addresses in the To: and Cc: headers of the incoming message are
looked up among the user nodes, and the corresponding users are placed in
the "recipients" property on the new "msg" node. The address in the From:
header similarly determines the "author" property of the new "msg"
node. The default handling for addresses that don't have corresponding
users is to create new users with no passwords and a username equal to the
address. (The web interface does not permit logins for users with no
passwords.) If we prefer to reject mail from outside sources, we can simply
register an auditor on the "user" class that prevents the creation of user
nodes with no passwords.

Actions
-------
The subject line of the incoming message is examined to determine
whether the message is an attempt to create a new item, discuss an
existing item, or execute some other command.

If the subject consists of one of the following (case insensitive),
the corresponding action is taken:

help
  Respond with an explanation of this interface.

If the subject contains the following, the corresponding action is
taken:

-- key <OneTimeKey>
  Complete an in-progress user registration.

If the email is not a command, it is either a new item or a message
associated with an existing item.  A designator enclosed in square
brackets is sought as the first thing on the subject line (after
skipping any "Fwd:" or "Re:" prefixes).

If an item designator (class name and id number) is found there, the newly
created "msg" node is added to the "messages" property for that item, and
any new "file" nodes are added to the "files" property for the item.

If just an item class name is found there, we attempt to create a new item
of that class with its "messages" property initialized to contain the new
"msg" node and its "files" property initialized to contain any new "file"
nodes.

Summary
-------
The "summary" property on message nodes is taken from the first non-quoting
section in the message body. The message body is divided into sections by
blank lines. Sections where the second and all subsequent lines begin with
a ">" or "|" character are considered "quoting sections". The first line of
the first non-quoting section becomes the summary of the message.

Triggers
--------
Both cases may trigger detectors (in the first case we are calling the
set() method to add the message to the item's spool; in the second case we
are calling the create() method to create a new node). If an auditor raises
an exception, the original message is bounced back to the sender with the
explanatory message given in the exception.

--===============0112997174240416991==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Return-Path: <[email protected]>
X-Original-To: [email protected]
Delivered-To: [email protected]
Received: from mail.priesch.priv.at (priesch.priv.at [193.238.157.80])
	by bugs.ams1.psf.io (Postfix) with ESMTPS id 5E3BC52B09A
	for <[email protected]>; Tue,  7 Jun 2022 09:56:18 +0000 (UTC)
Received: from ferdl.home (ferdl.home [10.3.0.2])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits))
	(No client certificate requested)
	by mail.priesch.priv.at (Postfix) with ESMTPS id 80B88347781
	for <[email protected]>; Tue,  7 Jun 2022 11:56:17 +0200 (CEST)
Received: from [192.168.4.101] (ruta.home [192.168.2.251])
	(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by ferdl.home (Postfix) with ESMTPSA id F154AE1E7F
	for <[email protected]>; Tue,  7 Jun 2022 11:56:16 +0200 (CEST)
Message-ID: <[email protected]>
Date: Tue, 7 Jun 2022 11:56:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.1
Subject: Re: [issue2551203] Add support for CORS preflight request
Content-Language: de-AT
To: Roundup tracker <[email protected]>
References: <[email protected]>
From: Marcus Priesch <[email protected]>
Organization: priesch.co.at - open source consulting
In-Reply-To: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi John,

sorry for not answering for that long, but i was off for sick-leave :(

Thanks for the patch, i have tested it and it works !

unsetting allowed_api_origins leads to a 400, setting it to * or
http://localhost:8081 (where the frontend server is running) works :)

so i think for this we can apply this patch and go ahead ;)

btw: as per your request, here are the headers from the preflight 
request as seen from devtools:

request:

OPTIONS 
/t/rest/data/time_activity?@sort=name&@verbose=0&@fields=name,id,description,travel&is_valid=1 
HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,de;q=0.8
Access-Control-Request-Headers: content-type,x-requested-with
Access-Control-Request-Method: GET
Cache-Control: no-cache
Connection: keep-alive
Host: localhost:8080
Origin: http://localhost:8081
Pragma: no-cache
Referer: http://localhost:8081/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, 
like Gecko) Chrome/101.0.4951.64 Safari/537.36

answer:

HTTP/1.1 204 No Content
Server: BaseHTTP/0.3 Python/2.7.18
Date: Tue, 07 Jun 2022 09:49:24 GMT
Access-Control-Allow-Methods: OPTIONS, GET, POST
Access-Control-Max-Age: 86400
Vary: Origin, Origin
Allow: OPTIONS, GET, POST
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost:8081
Access-Control-Allow-Headers: Content-Type, Authorization, 
X-Requested-With, X-HTTP-Method-Override


thanks for the nice work and good support !

regards,
marcus.

--===============0112997174240416991==--