Re: sendmail blackhole

Hi,

Perhaps there was some misunderstanding, my patch actually makes regex match
more lines not add more info to the hash.
For instance, current code does:

   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
/^ruleset=(?:check_relay|check_rcpt), arg1=([^,]*),(?: arg2=[^,]*,)?
reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)\//) ) {
      #Example 553 error with NO RELAY -mgt
      #ruleset=check_relay, arg1=s010600402b39ee29.vf.shawcable.net,
arg2=127.0.0.2, reject=553 5.3.0
      #Spam blocked see: http://spamcop.net/bl.shtml?70.68.8.182: 1 Time(s)

So it expects the sendmail message to include the full url:
"http://spamcop.net/bl.shtml?70.68.8.182: 1 Time(s)" including the
client_addr as explained in the comment.

This regex actually needs 3 slashes in the url to match:
"http:\/\/([^\/]*)\/" and assigns the string before the third slash to
$BlSite. Precisely because sendmail configuration can differ greatly and the
blacklist provider might not provide public url to full lookup with client
address, my suggestion was to also match lines where only site name is
displayed without the client_addr, so to end the regex with either '/ or
newline' match. The modified regex would match lines like: "- see
http://www.myblacklistprovider.net" (as an example) without the client_addr
at the end. In both cases only the domain name part is used in $BlSite.

I don't believe such a change would significantly impact sendmail reports in
the negative way, if I'm not mistaken.
Happy New Year!

Regards,
gulikoza


-----Original Message-----
From: Bjorn [mailto:[email protected]] 
Sent: Tuesday, December 30, 2014 10:04 PM
To: [email protected]
Subject: Re: [Logwatch-devel] sendmail blackhole


While I didn't write the blackhole reporting code in Logwatch sendmail, I am
familiar with this service script, so I wanted to provide some
thoughts:

I think the original intent was to have Logwatch identify the sources of the
blacklist (the sites from which the blacklist was obtained).

One issue is that this code is not only for spamcop, the blacklist provider
you are using, but any blacklist.  And there is the assumption that the
blacklist website is listed on the error message that is returned by the
sendmail MTA.  This error message is something that each user can configure
on its own in the pertinent mc file (or even the cf file).  Also, additional
variables can be inserted, such as the client_addr you are using on your
configuration file, making it very hard to know what the format or contents
of the error message is beforehand.

Placing the whole string in $BlSite might cause the Logwatch report to
display all this other information that may not be suitable for a summary,
and also break out individual error lines that otherwise would be reported
as a single summary line with a larger error count.

Looking at the Logwatch sendmail code, it appears that the Blackholed hash
is the one that gets displayed when $Detail >= 10, while the contents of the
BlackHoles hash is displayed at $Detail >= 5.  So if additional info is
added, it would seem preferable to use the BlackHoled hash only instead of
both BlackHoles and BlackHoled.  Actually, it would seem preferable to use a
hash of hashes, as was done with other error categories (such as
$BlackHoled{$BlSite}{$Relay}).

[As an added comment, it seems there is some commingling on checking of
check_relay and check_rcpt; and arg2 and relay; but it appears that the
resulting report displays the available data.]

In summary, I think that the patch suggested may adversely impact other
people's Logwatch sendmail reports.  There might be a more universal
solution, such as putting additional info in the BlackHoled hash only (or in
hash of hashes), but I don't have blacklist error reports to check or test.

As a final note, I'll mention $MatchFilter and $ReportFilter, which are
described in the configuration file.  It allows individual sites to
customize the sendmail report, precisely because the sendmail MTA allows so
much customization, whether through changes in the mc file, the cf file, or
the many milters available.  These two variables allow changing the Logwatch
sendmail script without having to patch it every time a new sendmail script
is released.


Bjorn



On 12/28/2014 08:28 AM, gulikoza wrote:
> Hi,
> 
> Since sendmail seems to be on the spot, would it be possible to modify 
> blackhole regex not to search for the third / at the end of the url.
> My sendmail only prints sitename ("- see http://www.spamcop.net") not 
> the full lookup url ("http://spamcop.net/bl.shtml?70.68.8.182"). 
> Perhaps something like (match [\/\n] instead on \/):
> 
> --- sendmail.org        2014-12-28 16:08:30.472681423 +0100
> +++ sendmail    2014-12-28 16:18:28.294860546 +0100
> @@ -823,18 +823,18 @@
>        $Temp = "From " . $1 . " by " . $2;
>        $BlackHoled{$Temp}++;
>        $BlackHoles{$2}++;
> -   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
> /^ruleset=(?:check_relay|check_rcpt), arg1=[^,]*,(?: arg2=[^,]*,)?
> relay=([^,]*), reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)\//) ) {
> +   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
> /^ruleset=(?:check_relay|check_rcpt), arg1=[^,]*,(?: arg2=[^,]*,)?
> relay=([^,]*), reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)[\/\n]/) ) {
>        $Temp = "From " . $Relay . " by " . $BlSite;
>        $BlackHoled{$Temp}++;
>        $BlackHoles{$BlSite}++;
> -   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
> /^ruleset=(?:check_relay|check_rcpt), arg1=([^,]*),(?: arg2=[^,]*,)?
> reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)\//) ) {
> +   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
> /^ruleset=(?:check_relay|check_rcpt), arg1=([^,]*),(?: arg2=[^,]*,)?
> reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)[\/\n]/) ) {
>        #Example 553 error with NO RELAY -mgt
>        #ruleset=check_relay, arg1=s010600402b39ee29.vf.shawcable.net,
> arg2=127.0.0.2, reject=553 5.3.0
>        #Spam blocked see: http://spamcop.net/bl.shtml?70.68.8.182: 1
Time(s)
>        $Temp = "From " . $Relay . " by " . $BlSite;
>        $BlackHoled{$Temp}++;
>        $BlackHoles{$BlSite}++;
> -   } elsif ( ($Relay,$BlSite) = ($ThisLine =~ /reject=553\s*[\d.]*\s*<[^
> ]*>\.\.\. +Mail from ([\d\.]+) rejected\;see http:\/\/([^\/]*)\//) ) {
> +   } elsif ( ($Relay,$BlSite) = ($ThisLine =~ 
> + /reject=553\s*[\d.]*\s*<[^
> ]*>\.\.\. +Mail from ([\d\.]+) rejected\;see http:\/\/([^\/]*)[\/\n]/) ) {
>        #This is the another blackhole tag -mgt
>        $Temp = "From " . $Relay . " by " . $BlSite;
>        $BlackHoled{$Temp}++;
> 
> 
> ----------------------------------------------------------------------
> -------- Dive into the World of Parallel Programming! The Go Parallel 
> Website, sponsored by Intel and developed in partnership with Slashdot 
> Media, is your hub for all things parallel software development, from 
> weekly thought leadership blogs to news, videos, case studies, 
> tutorials and more. Take a look and join the conversation now. 
> http://goparallel.sourceforge.net 
> _______________________________________________
> Logwatch-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/logwatch-devel
> 


----------------------------------------------------------------------------
--
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
Logwatch-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/logwatch-devel


------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net