Re: Lots of FN because of VALIDITY* rules

On 6/3/24 12:02, Matus UHLAR - fantomas wrote:
 > On 03.06.24 07:26, postgarage Graz IT wrote:
 >> A few days ago a lot of false negatives landed in our inboxes. As it
 >> turned out the reason was that the for nearly all mails the
 >> RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE rules matched.
 >>
 >> I now know that validity introduced a query limit which we hit,
 >> because I have to admit, I wasn't aware that I shouldn't use public
 >> DNS resolvers for blacklists
 >
 > I'd say you should not use public DNS resolvers with mailserver.

Thanks. I know that by now and already set up a local DNS resolver. But 
that's not been the problem.

 >> and therefore we got "Excessive Number of Queries" answers. I also
 >> found this patch
 >> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8244 which
 >> introduces new rules addressing the query limit.
 >
 > my current rules show that all RCVD_IN_VALIDITY_* rules check for 
blocked.

The rules were here. The issue was, that they weren't active, as the 
were missing from the active.list file. There's no active.list under 
/var/lib/spamassassin/4.000000/updates_spamassassin_org/. There's  only 
one in /usr/share/spamassassin/, which is provided by the debian 
package. But within that file the new *BLOCKED rules aren't activated.

So the situation was:
*) The updated *VALIDITY* rules were active
*) the new *VALIDITY*BLOCKED rules weren't active

Which lead to almost every mail passing the spamfilter, as for every 
"Excessive Number of Queries" answer from validity 
RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE counted with -5 to 
the score.

AFTER I manually added the *BLOCKED rules to 
/usr/share/spamassassin/active.list I get the correct results
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, 
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,

I think that the active.list file should be updated, when there are new 
rules, shouldn't it?


 >> Those *BLOCKED rules where never applied because our spamassassin
 >> received an updated rule-set which was saved to
 >> /var/lib/spamassassin/4.000000/updates_spamassassin_org/ but never
 >> received an update for the active.list file located in
 >> /usr/share/spamassassin/
 >
 >> After I manually added the changes from the above mentioned patch to
 >> the active.list file it started to work.
 >>
 >> Now for my questions:
 >> *) as is stated in active.list it should not be edited. What's the
 >> correct place to add the new rules to activate them? local.cf?
 >
 > you can use dns_query_restriction to restrict which DNS lists to query.
 >
 > further, you can tune uridnsbl_skip_domain to avoid lookups for domains
 > in URI* lists.
 >
 >> *) If I understand it correctly
 >> /var/lib/spamassassin/4.000000/updates_spamassassin_org/ is updated by
 >> the SA update mechanism but it's the Linux distribution's
 >> responsibility to update /var/lib/spamassassin? In that case should I

Sorry, that's a mistake by my side, should have been 
/usr/share/spamassassin/

 >> fill a Debian bug? Or should the SA updates also include the file
 >> active.list?
 >
 > reload spamd or amavis, the rules in /var/lib/spamassassin/ are used by
 > default.
 >
 > Maybe you need to enable cron job by setting CRON=1 in
 > /etc/default/spamassassin and it will happen automatically.
 >
 > ...I have no idea how active.list works.
 >
 >