Re: Dealing with spam on the mailing lists

On Tue, 24 Dec 2019, [email protected] wrote:

> On Thu, 19 Dec 2019 22:11:32 +0000
> Steve McIntyre <[email protected]> wrote:
> 
> > Hey folks,
> 
> Hello all!
> 
> > So, I've spoken to the listmaster team about making this list
> > "moderated" rather than "open". What does that mean? Mails from
> > subscribers would go through to the list; mails from non-subscribers
> > would be held back for moderation by humans. We'd need some
> > responsible people to act as moderators, and I volunteer to help
> > here.
> 
> > What do you think? Is this the right thing to do?
> 
> I feel that moderating the list, out of a ton of debian MLs, is taking a
> very localized view on the problem. Certainly, other MLs could follow
> the path to moderation; but I feel the human-resource drain for
> moderation seems significant, and interest in doing the work - possibly
> temporary.
> Of course it looks quite necessary in the face of not *having* a
> workable solution, but it would be hiding the bigger issue from public
> view, at best, so I'd prefer an automated solution, if possible.
> 
> Human moderation of every unsubbed email, continuously, looks to me like
> an uphill battle against resource deprivation.
> And if the mod stops moderating (for lack of time, etc.), we wouldn't
> even notice until a non-subscriber eventually complains through a
> different channel? Additionally, delays due to TZ-differences,
> availability etc. of the moderators, are likely to happen.
> 
> But we do know blacklisting is doomed to fail, and a whitelist solution,
> like moderation, seems like the only real answer in today's email
> landscape. The only question is of it's feasibility in the given
> environment.
> 
> Here is my off-the-cuff response to the problem. It looks feasible to
> me, but without knowing the debian ML situation intimately, I can't
> tell for certain. So far it is only a thought-experiment.
> 
> 
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> 
> I propose a global double-opt-in approach for unknown email-addresses
> to debian's Mailing-Lists as a non-human moderation process.
Several spammers subscribe. 

> 
> This system would replace the currently optional
> <whitelist et lists.d.o> system (https://lists.debian.org/whitelist/)
> with a _required_ opt-in. (And it could start by importing subscribed
> addresses from that list.)
Several spammers do that too. 

> 
> Consider it a "lightweight" registration process, an opt-in without
> at the same time subscribing to any specific debian ML:
> 
> 
> [1] The first time an email address is seen by the ML-system, bendel, a
>  |  confirmation-request mail is sent back (the double opt-in), to
>  |  confirm the validity of the address, as well as the sender's
>  |  intent.
>  |  This is done using a canned email-template, which can be
>  |  replied to just like any response from <*[email protected]>.
>  |  It could include a reminder of the ML CoC, do and dont's[1].
> 
> [2] At this point the address is stored in a lookup-table on the
>  |  Mail-system -for all MLs- as "pending" or "greylisted" until the
>  |  opt-in mail is confirmed. (So that further mails from this possibly
>  |  spammy address to any debian ML don't result in backscatter spam
>  |  by repetitive confirm requests to some unsuspecting address).
>  |
>  |  Further, the initially received mail (and any further mail sent
>  |  in the meantime) is placed on hold in the mailer queue, and
>  |  released upon opt-in confirmation from the greylisted address.
>  |
>  |  [2b] After some time has elapsed without confirmation (perhaps a
>  |  week), the held mail gets zapped by a cronjob cleanup run or some
>  |  such; and the address times out from the "greylist" lookup table.
>  |
>  |  This greylist measure should probably run after any DNSBL and
>  |  other spam-checks (such as SA and greylisting) already weeded out
>  |  the chaff, just before the mail would have been forwarded to
>  |  SmartList. (This should mean the hold queue would have approximately
>  |  the amount of mails currently seen on the lists as spam, per week.)
> 
> [3] Once the confirmation reply arrives, the email address is moved
>  |  over to a global whitelist table, which removes the need for new
>  |  opt-in confirmations from this email address, when sending to
>  |  *any* debian ML, from then on.
>  |
>  |  Further, the hold on all mail from this address in the mailer queue
>  |  is released.
>  |
>  |  [3b] The whitelist table lookup shouldn't be very resource
>  |  intensive, so it could run before some of the other content checks,
>  |  and a positive hit could negate the need for the more
>  |  resource-intensive SA run, thus taking load off the system.
>  |  (If that feels too permissive, it could at least be a powerful
>  |  weight in the SA rules.)
> 
> [4] Should a stored mail address ever become spammy, it would need to
>  |  be removed from the whitelist. This should be the only task
>  |  requiring human intervention in this system (as opposed to
>  |  moderation / per email / per list). But maybe I'm overly optimistic.
> 
> 
> In terms of implementation, I feel this could be a lightweight
> solution. It could be written as a postfix access table check and
> milter; or in the case of exim here(?), a milter program would work for
> that MTA as well, I believe?
I fear that this will not help a lot or at least not for a long time. 

Alex - Debian Listmaster