mod_backhandunresponsive, then failure to requests

Lister <[email protected]> Sun, 20 Feb 2005 17:16:51 -0800
Newsgroups gmane.comp.apache.mod-backhand.general
Message-ID <[email protected]>
 Configuration :
10.200.1.111 and 10.200.1.112
OS : 4.11-STABLE FreeBSD
packages (all source compiled from ports) :
apache+mod_ssl-1.3.33+2.8.22
mod_backhand-1.2.2 
mod_bandwidth-2.0.5
mod_dav-1.0.3_2    
mod_php4-4.3.10_2,1

 Node 1 httpd.conf :
<IfModule mod_backhand.c>
        UnixSocketDir /var/backhand/backhand
        MulticastStats 10.200.1.111 10.200.1.255:4445,1
        AcceptStats 10.200.1.0/24
</IfModule>

<Location "/backhand/">
    SetHandler backhand-handler
</Location>

<Directory /usr/local/www/data/www.lauraandamber.com>
        Backhand removeSelf
       Backhand byBusyChildren
</Directory>

 Node 2 httpd/conf :
<IfModule mod_backhand.c>
    UnixSocketDir /var/backhand/backhand
                MulticastStats 10.200.1.112 10.200.1.255:4445,1
        AcceptStats 10.200.1.0/24
</IfModule>

<Location "/backhand/">
    SetHandler backhand-handler
</Location>
Both with and with out this :
<Directory /usr/local/www/data/www.lauraandamber.com>
        Backhand addSelf
</Directory>

 I (re)start apache and see
[Sun Feb 20 17:10:17 2005] [notice] SIGUSR1 received.  Doing graceful 
restart
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand -- UnixSocketDir set to 
/var/backhand/backhand
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand -- Broadcast 
10.200.1.255:4445 added
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand -- Multicast accept 
10.200.1.0/24
[Sun Feb 20 17:10:17 2005] [notice] backhand_init(4859) spawning 
moderator (PID 4910)
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand moderator ready to go
[Sun Feb 20 17:10:18 2005] [notice] Apache/1.3.33 (Unix) PHP/4.3.10 
mod_backhand/1.2.2 DAV/1.0.3 configured -- resuming normal operations
[Sun Feb 20 17:10:18 2005] [notice] Accept mutex: flock (Default: flock)

 After a bunch of refreshes normal (I see the pages in each system's
access log) to the page :
http://bookpc1/www.lauraandamber.com/
 It stops responding, and I see (something like) this :

 > netstat
tcp4       0      0  10.200.1.111.4179      10.200.1.112.http      SYN_SENT
tcp4       0      0  10.200.1.111.3412      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.1875      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4199      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.2039      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3527      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4922      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4064      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3785      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4986      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3011      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.1702      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.2660      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3068      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4948      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.2797      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3074      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4851      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4449      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3046      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4777      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.http      10.200.1.90.4496       TIME_WAIT
tcp4       0      0  10.200.1.111.4098      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3982      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.1582      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.1488      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.1329      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.http      10.200.1.90.1974       TIME_WAIT
tcp4       0      0  10.200.1.111.4357      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.2882      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.2489      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3332      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.4193      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3413      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3676      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.http      10.200.1.90.1715       TIME_WAIT
tcp4       0      0  10.200.1.111.http      10.200.1.90.1416       TIME_WAIT
tcp4       0      0  10.200.1.111.4907      10.200.1.112.http      TIME_WAIT
tcp4       0      0  10.200.1.111.3817      10.200.1.112.http      TIME_WAIT

I see nothing in the back end's (bookpc2, .112) logs, but in the front
end's logs 9bookpc1, .111) I start to (slowly) see these :
[Sun Feb 20 16:29:55 2005] [warn] (4)Interrupted system call: connect() 
timed out

 Eventually, I see this :
[Sun Feb 20 14:18:09 2005] [error] [client 10.200.1.90] Request exceeded 
the limit of 20 internal redirects due to probable configuration error. 
Use 'LimitInternalRecursion' to increase the limit if necessary. Use 
'LogLevel debug' to get a backtrace.

 And / or this :
[Sun Feb 20 16:53:12 2005] [error] mod_backhand: child 4734 hung talking 
to moderator. Die!
 And
[Sun Feb 20 17:06:41 2005] [error] (61)Connection refused: Child 4874 
failed to establish umbilical to moderator!
 And
[Sun Feb 20 17:08:09 2005] [notice] child pid 4860 exit signal Broken 
pipe (13)
 And
message format error

 This list / package appears to be have relatively low volume of
chatter. This could either mean 'really stable' or 'dead project'.
I asked on #apache, and people said it was 'in use, but not being
developed (for)'.
 I would love to use this to load balance for some XML-RPC scripts
I am developing, but it has to berock solid for this. If this is not the
tool, is there something that people are using now?

 TIA