Re: Fw: Fairly critical issue with apache logging to error_log_spread.pl

Hi Jay,

Ian Holsman has ported m_l_s to apache2.  The port is in the m_l_s cvs 
repository in the apache2 branch.


Regarding the script, here's a small change that has it disconnect and 
reconnect in the event of a multicast failure.  

use Spread qw(:SP :MESS :ERROR);
use Getopt::Std;
$| = 1;

# Read in options
getopts('dg:hs:', \%opts);
$debug = $opts{d};
&usage() unless ($group = $opts{g});
$hosttoggle = $opts{h};
$spreaddaemon = ($opts{s}?$opts{s}:3333);


# Set spread connection params
#$hostname = `hostname`;
chomp ($hostname = `hostname`) ;
$args{'spread_name'} = $spreaddaemon;
$args{'private_name'} = "$$-$hostname";
$args{'priority'} = 0;
$args{'group_membership'} = 1;

# Connect to daemon
connect();
#Logging loop
while(<STDIN>){
    chomp;
    print "MESSAGE $_\n";
    if(($ret = Spread::multicast($mbox,
                             RELIABLE_MESS, $group, 0,
                             ($hosttoggle?$hostname." ".$_:$_)))>0) {
        print STDERR "Successfully multicasted $ret bytes to [$group]\n" 
if $debug;
    } else {
        print STDERR "Failed multicast $_ to $group: $sperrno\n" if $debug;
        connect();
    }
}

sub usage(){
        print STDERR "Usage:  Logger -g group [-h] [-s daemon] [-d]\n";
        exit 0;
}

sub connect {
    print "Trying to connect to spread...\n" if $debug;
    Spread::disconnect($mbox) if $mbox;
    ($mbox, $privategroup) = Spread::connect(\%args);
    print "$sperrno\n" unless (defined($mbox) && $debug);


Jay West wrote:

>>Greetings folks:
>>
>>Since we are using Apache2, we can't use mod_log_spread... at least
>>
>directly
>
>>and officially in production - yet. However, we are making use of just a
>>portion of it - specifically the included file "error_log_spread.pl". This
>>small perl script is really intended for logging apache errors, but we use
>>it for the transferlog so that all transfer logs go to a central host from
>>multiple websites. For example, in most of our virtual host setups on
>>apache, we have...
>>
>>CustomLog "|/usr/local/sbin/error_log_spread.pl -g hdcoutlet" combined
>>
>>We have made one small customization to the stock error_log_spread.pl
>>program - any messages it receives via chomp have the trailing LF removed,
>>so this one section of code is now...
>>
>>if(($ret = Spread::multicast($mbox,
>>                             RELIABLE_MESS, $group, 0,
>>                             ($hosttoggle?$hostname." ".$_:$_."\n")))>0) {
>>  print STDERR "Successfully multicasted $ret bytes to [$group]\n" if
>>$debug;
>>} else {
>>  print STDERR "Failed multicast $_ to $group: $sperrno\n" if $debug;
>>}
>>
>>Note specifically the addition of "\n". This gets us the original log line
>>written back out with the LF "intact".
>>
>>On to the problem - we have probably 15 or 20 virtual hosts that use the
>>above CustomLog statement, piping through error_log_spread.pl, with each
>>site having a different log group so spreadlogd can put the logs in
>>different files. All is well and good, this works great. However, whenever
>>we do an "apachectl stop" and then "apachectl start", sometimes, but not
>>
>all
>
>>the time, SOME of the error_log_spread.pl pipes print up "Could Not
>>Connect". It is kind of a random thing. If we do an apachectl stop and
>>apachectl start again, most of the time they all start right (no
>>
>messages),
>
>>but sometimes a few less fail, do it again, then none fail. I am making a
>>wild guess that this is some kind of timing issue with 15 or so pipes
>>
>coming
>
>>up trying to connect to spread. This becomes very problematic for us,
>>because our developers release code with cvs in an automated fashion. They
>>can "goose" a machine that will do a cvs export, then that machine will
>>rsync the website code modules to all the webservers in the wackamole
>>cluster. In order to keep the sites up while this is going on, this
>>"controlling" machine needs to send ssh commands to the webservers, one at
>>
>a
>
>>time, to do an apachectl stop, then a wackatrl -f, (the load is all on
>>
>other
>
>>webservers in the wackamole cluster now), then it rsyncs the website code,
>>then a wackatrl -s, then an apachectl start. It then does this in turn to
>>each machine in the wackamole cluster. Of course, it's a real problem if
>>when the apachectl start is done, that the logs dont come up because of a
>>"could not connect" for just a few of the sites.
>>
>>Perhaps (wild uninformed guess here) all that needs to be done is for the
>>error_log_spread.pl program to be changed to try to connect to spread
>>several times, rather than just trying once and giving up. I'm not a perl
>>programmer - if this is the issue, could someone show how the script
>>
>should
>
>>be changed to do it? If this isn't the likely issue or best solution - any
>>ideas??
>>
>>Thanks!
>>
>>Jay West
>>
>
>---
>[This E-mail scanned for viruses by Declude Virus]
>
>
>_______________________________________________
>mls-users mailing list
>[email protected]
>http://lists.backhand.org/mailman/listinfo/mls-users
>