Re: ldirectord spinning on SIGPIPE

On Mon, Jan 12, 2009 at 07:26:01PM +0000, Matthew Newton wrote:
> Hi,
> 
> On Mon, Jan 12, 2009 at 08:49:10PM +1100, Simon Horman wrote:
> > On Mon, Jan 05, 2009 at 01:13:57PM +0000, Matthew Newton wrote:
> > > Recently the ldirectord process has stopped working fairly
> > > frequently. Symptoms are that services are not being checked, and
> > > the ipvs tables are not updated. The ldirectord process is still
> > > running, but stracing it gives, for example:
> > > 
> > > --- SIGPIPE (Broken pipe) @ 0 (0) ---
> > > write(4, "\27\3\1\0\33\r\310Q\347\n\230\375@\370 \372\263\322\256"..., 32) = -1 EPIPE (Broken pipe)
> > 
> > Almost certianly signal-handler badness = bug.
> > 
> > I strongly suspect that this has been fixed in more recent versions of
> > ldirectord (I recall fixing it). I can try and verify that if it helps.
> 
> OK, thanks - I'll try the latest version first! It's been running
> all week with strace, and no sign of a problem, annoyingly.

Hi Matthew,

I had a poke into the change log and found the commit that I was thinking
about. It is below. However it seems to have been applied well
before the release of 2.0.7 - the patch was applied in May 2004
and 2.0.7 was released in August 2006. So unfortuantely I think
that you have found a new problem that is likely to still be present
in the latest version.

I guess that what is happening is this:

  1. attempt to write to a socket/fd
  2. recieve sigpipe - probably because the socket is closed
  3. but sigpipe is ignored, so try the write again. repeat endlessly...

If that is correct, then it seems that simply ignoring sigpipe
is not a good idea.  I'll have to shift my mind into perl-mode
in order to think about what to do more clearly.

> Is the latest 2.1.4 ldirectord a drop-in replacement for 2.0.8? If
> so, I'll do so next time it crashes (or some other time if it
> doesn't...).

Yes, it should be.

> I like the look of the new "fork" option - that was going to be my
> next question! Checking is very slow at the moment due to the
> number of services.

-- 
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en


# HG changeset patch
# User horms
# Date 1085970444 0
# Node ID e9f9f7bdd634938ebc03b92ff003a8509f62ea3d
# Parent  25492d91c074c9ed7f5e41eb79d3389c1f07e036
# URL http://hg.linux-ha.org/dev/rev/e9f9f7bd
Don't log on SIGPIPE, because that can cause a SIGPIPE!

CVS patchset: 3147
CVS date: 2004/05/31 02:27:24

diff -r 25492d91c074 -r e9f9f7bdd634 ldirectord/ldirectord
--- a/ldirectord/ldirectord	Fri May 28 07:55:28 2004 +0000
+++ b/ldirectord/ldirectord	Mon May 31 02:27:24 2004 +0000
@@ -1,5 +1,5 @@
 #!/usr/bin/perl
-my($VERSION)=(qw$Id: ldirectord,v 1.88 2004/05/27 05:29:05 horms Exp $)[2];
+my($VERSION)=(qw$Id: ldirectord,v 1.89 2004/05/31 02:27:24 horms Exp $)[2];
 
 ######################################################################
 # ldirectord                 http://www.vergenet.net/linux/ldirectord/
@@ -443,7 +443,10 @@
 	$SIG{'__WARN__'} = "DEFAULT";
 
 	# handle PIPE separately as it probably should be ignored
-	$SIG{'PIPE'} = \&ld_handler_ignore;
+	# This used to call a signal handler, that logged a message
+	# However, this typically goes to syslog and if syslog
+	# is playing up a loop will occur. 
+	$SIG{'PIPE'} = "IGNORE";
 
 	# HUP is actually used
 	$SIG{'HUP'} = \&ld_handler_hup;
@@ -567,12 +570,6 @@
 }
 
 
-sub ld_handler_ignore
-{
-	my ($signal) = (@_);
-	&ld_log("Ignoring signal: $signal");
-	
-}
 sub ld_handler_hup
 {
         my ($signal) = (@_);
_______________________________________________
Ultramonkey-users mailing list
[email protected]
http://lists.vergenet.net/listinfo/ultramonkey-users