Re: sendmail blackhole

Even better. Works for me!

Thank you,
gulikoza

-----Original Message-----
From: Bjorn [mailto:[email protected]] 
Sent: Saturday, January 03, 2015 11:34 PM
To: [email protected]
Subject: Re: [Logwatch-devel] sendmail blackhole

Gulikoza,

Got it.  In that case, I'd like to suggest searching for [^\/\s], which will
include any characters that are not a slash nor whitespace, which also
applies to newline.  That will also take into account the form
	http://example.com info
where info is additional string after the URL and a space or a tab.

Attached is the patch against the repository version of sendmail.  Can you
let me know if that works?


On 12/31/2014 09:50 AM, gulikoza wrote:
> Hi,
> 
> Perhaps there was some misunderstanding, my patch actually makes regex 
> match more lines not add more info to the hash.
> For instance, current code does:
> 
>    } elsif ( ($Relay,$BlSite) = ($ThisLine =~ 
> /^ruleset=(?:check_relay|check_rcpt), arg1=([^,]*),(?: arg2=[^,]*,)?
> reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)\//) ) {
>       #Example 553 error with NO RELAY -mgt
>       #ruleset=check_relay, arg1=s010600402b39ee29.vf.shawcable.net,
> arg2=127.0.0.2, reject=553 5.3.0
>       #Spam blocked see: http://spamcop.net/bl.shtml?70.68.8.182: 1 
> Time(s)
> 
> So it expects the sendmail message to include the full url:
> "http://spamcop.net/bl.shtml?70.68.8.182: 1 Time(s)" including the 
> client_addr as explained in the comment.
> 
> This regex actually needs 3 slashes in the url to match:
> "http:\/\/([^\/]*)\/" and assigns the string before the third slash to 
> $BlSite. Precisely because sendmail configuration can differ greatly 
> and the blacklist provider might not provide public url to full lookup 
> with client address, my suggestion was to also match lines where only 
> site name is displayed without the client_addr, so to end the regex 
> with either '/ or newline' match. The modified regex would match lines 
> like: "- see http://www.myblacklistprovider.net" (as an example) 
> without the client_addr at the end. In both cases only the domain name
part is used in $BlSite.
> 
> I don't believe such a change would significantly impact sendmail 
> reports in the negative way, if I'm not mistaken.
> Happy New Year!
> 
> Regards,
> gulikoza
> 
> 
> -----Original Message-----
> From: Bjorn [mailto:[email protected]]
> Sent: Tuesday, December 30, 2014 10:04 PM
> To: [email protected]
> Subject: Re: [Logwatch-devel] sendmail blackhole
> 
> 
> While I didn't write the blackhole reporting code in Logwatch 
> sendmail, I am familiar with this service script, so I wanted to 
> provide some
> thoughts:
> 
> I think the original intent was to have Logwatch identify the sources 
> of the blacklist (the sites from which the blacklist was obtained).
> 
> One issue is that this code is not only for spamcop, the blacklist 
> provider you are using, but any blacklist.  And there is the 
> assumption that the blacklist website is listed on the error message 
> that is returned by the sendmail MTA.  This error message is something 
> that each user can configure on its own in the pertinent mc file (or 
> even the cf file).  Also, additional variables can be inserted, such 
> as the client_addr you are using on your configuration file, making it 
> very hard to know what the format or contents of the error message is
beforehand.
> 
> Placing the whole string in $BlSite might cause the Logwatch report to 
> display all this other information that may not be suitable for a 
> summary, and also break out individual error lines that otherwise 
> would be reported as a single summary line with a larger error count.
> 
> Looking at the Logwatch sendmail code, it appears that the Blackholed 
> hash is the one that gets displayed when $Detail >= 10, while the 
> contents of the BlackHoles hash is displayed at $Detail >= 5.  So if 
> additional info is added, it would seem preferable to use the 
> BlackHoled hash only instead of both BlackHoles and BlackHoled.  
> Actually, it would seem preferable to use a hash of hashes, as was 
> done with other error categories (such as $BlackHoled{$BlSite}{$Relay}).
> 
> [As an added comment, it seems there is some commingling on checking 
> of check_relay and check_rcpt; and arg2 and relay; but it appears that 
> the resulting report displays the available data.]
> 
> In summary, I think that the patch suggested may adversely impact 
> other people's Logwatch sendmail reports.  There might be a more 
> universal solution, such as putting additional info in the BlackHoled 
> hash only (or in hash of hashes), but I don't have blacklist error reports
to check or test.
> 
> As a final note, I'll mention $MatchFilter and $ReportFilter, which 
> are described in the configuration file.  It allows individual sites 
> to customize the sendmail report, precisely because the sendmail MTA 
> allows so much customization, whether through changes in the mc file, 
> the cf file, or the many milters available.  These two variables allow 
> changing the Logwatch sendmail script without having to patch it every 
> time a new sendmail script is released.
> 
> 
> Bjorn
> 
> 
> 
> On 12/28/2014 08:28 AM, gulikoza wrote:
>> Hi,
>>
>> Since sendmail seems to be on the spot, would it be possible to 
>> modify blackhole regex not to search for the third / at the end of the
url.
>> My sendmail only prints sitename ("- see http://www.spamcop.net") not 
>> the full lookup url ("http://spamcop.net/bl.shtml?70.68.8.182").
>> Perhaps something like (match [\/\n] instead on \/):
>>
>> --- sendmail.org        2014-12-28 16:08:30.472681423 +0100
>> +++ sendmail    2014-12-28 16:18:28.294860546 +0100
>> @@ -823,18 +823,18 @@
>>        $Temp = "From " . $1 . " by " . $2;
>>        $BlackHoled{$Temp}++;
>>        $BlackHoles{$2}++;
>> -   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
>> /^ruleset=(?:check_relay|check_rcpt), arg1=[^,]*,(?: arg2=[^,]*,)?
>> relay=([^,]*), reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)\//) ) {
>> +   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
>> /^ruleset=(?:check_relay|check_rcpt), arg1=[^,]*,(?: arg2=[^,]*,)?
>> relay=([^,]*), reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)[\/\n]/) ) {
>>        $Temp = "From " . $Relay . " by " . $BlSite;
>>        $BlackHoled{$Temp}++;
>>        $BlackHoles{$BlSite}++;
>> -   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
>> /^ruleset=(?:check_relay|check_rcpt), arg1=([^,]*),(?: arg2=[^,]*,)?
>> reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)\//) ) {
>> +   } elsif ( ($Relay,$BlSite) = ($ThisLine =~
>> /^ruleset=(?:check_relay|check_rcpt), arg1=([^,]*),(?: arg2=[^,]*,)?
>> reject=55\d\s*[\d.]*\s*.*http:\/\/([^\/]*)[\/\n]/) ) {
>>        #Example 553 error with NO RELAY -mgt
>>        #ruleset=check_relay, arg1=s010600402b39ee29.vf.shawcable.net,
>> arg2=127.0.0.2, reject=553 5.3.0
>>        #Spam blocked see: http://spamcop.net/bl.shtml?70.68.8.182: 1
> Time(s)
>>        $Temp = "From " . $Relay . " by " . $BlSite;
>>        $BlackHoled{$Temp}++;
>>        $BlackHoles{$BlSite}++;
>> -   } elsif ( ($Relay,$BlSite) = ($ThisLine =~ /reject=553\s*[\d.]*\s*<[^
>> ]*>\.\.\. +Mail from ([\d\.]+) rejected\;see http:\/\/([^\/]*)\//) ) 
>> {
>> +   } elsif ( ($Relay,$BlSite) = ($ThisLine =~ 
>> + /reject=553\s*[\d.]*\s*<[^
>> ]*>\.\.\. +Mail from ([\d\.]+) rejected\;see http:\/\/([^\/]*)[\/\n]/) )
{
>>        #This is the another blackhole tag -mgt
>>        $Temp = "From " . $Relay . " by " . $BlSite;
>>        $BlackHoled{$Temp}++;
>>
>>
>> ---------------------------------------------------------------------
>> -
>> -------- Dive into the World of Parallel Programming! The Go Parallel 
>> Website, sponsored by Intel and developed in partnership with 
>> Slashdot Media, is your hub for all things parallel software 
>> development, from weekly thought leadership blogs to news, videos, 
>> case studies, tutorials and more. Take a look and join the conversation
now.
>> http://goparallel.sourceforge.net
>> _______________________________________________
>> Logwatch-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/logwatch-devel
>>
> 
> 
> ----------------------------------------------------------------------
> ------
> --
> Dive into the World of Parallel Programming! The Go Parallel Website, 
> sponsored by Intel and developed in partnership with Slashdot Media, 
> is your hub for all things parallel software development, from weekly 
> thought leadership blogs to news, videos, case studies, tutorials and 
> more. Take a look and join the conversation now. 
> http://goparallel.sourceforge.net 
> _______________________________________________
> Logwatch-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/logwatch-devel
> 



------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net