Re: ldirectord memory leak in Ultramonkey 3

#I'm sorry for my poor English in advance. 

I also have the same trouble "ldirectord memory leak in UltraMonkey3"
in  a similar environment. 
So I have investigated the cause of the memory leak in various environment.
The environment that the memory leak has occured is common of
the following points.

- Perl version: 5.8.4, (also 5.8.5)
   It seems that the memory leak doesn't occur only Perl v5.8.4-8 in Debian Sarge
   but also Perl v5.8.4-? in RHEL3, v5.8.5-9, v5.8.5-24 in RHEL4.
- ldirectord version: ldirectord 1.2.3.cvs.20050927-*
- "checktype=connect  (in ldirectord.cf) ", namely in running check_connect()
   subroutine in ldirectord.
   It seems that the memory leak doesn't occur in "checktype=negotiate (default)".
   But  it might directly be unrelated to checktype.

As a result of some investigations, the cause of the memory leak seems to be related to
"Perl" rather than ldirectord (Perl application). Particularly, a memory management in Perl.
Then, referring to CPAN, the latest stable version of Perl is 5.8.8.
And referring to the Perl Delta Pods (Perl Release Notes) on CPAN site, 
it seems that various memory leaks (delete, pop, List::Util and threads) were fixed
in v5.8.6, v5.8.8. 
http://perldoc.perl.org/perl586delta.html
http://perldoc.perl.org/perl588delta.html
I think that the memory leak in ldirectord  is probably related to pop (@_ in subroutine) 
or delete etc fixed in v5.8.6. (but I haven't investigated them in detail yet.)

As I tried Perl v5.8.8 previously from v5.8.6 on my personal environment,  I didn't try v5.8.6.
(my PC: RHEL4 ES, Perl v5.8.5-9(default); So I compiled and installed the perl v5.8.8 source code.)  
As far as I tested the ldirectord using Perl v5.8.8, the memory leak didn't occur.

Would you try to update Perl v5.8.4 -> v5.8.6 -> v5.8.8 if you can, 
and check the memory use rate about  the ldirectord, Joseph Miller etc ?

--
Hideaki Kondo (from Japan)

----- Original Message ----- 
From: "Joseph Miller" <[email protected]>
To: <[email protected]>
Sent: Monday, October 17, 2005 4:44 AM
Subject: Re: ldirectord memory leak in Ultramonkey 3


> I installed ldirectord_1.2.3.cvs.20050927-1bpo1_all.deb from that URL above, and am now seeing this 
> in 'top':
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 12244 root      16   0 88920  84m 3832 S  2.0 33.9  46:22.73 ldirectord
> 
> I'm running Debian Sarge, with perl v5.8.4, as per default install under Debian
> 
> ldirectord.cf is below (external IPs masked to protect the innnocent):
> 
> checktimeout=10
> checkinterval=1
> fallback=127.0.0.1:80
> autoreload=yes
> logfile="/var/log/ldirectord.log"
> logfile="local0"
> quiescent=no
> 
> virtual=XXX.XXX.XXX.100:80
>         fallback=127.0.0.1:80
>         real=192.168.50.4:80 masq
>         real=192.168.50.5:80 masq
>         scheduler=rr
>         checktype=connect
>         protocol=tcp
> 
> virtual=XXX.XXX.XXX.100:443
>         fallback=127.0.0.1:80
>         real=192.168.50.4:443 masq
>         real=192.168.50.5:443 masq
>         scheduler=rr
>         checktype=connect
>         protocol=tcp
> 
> virtual=XXX.XXX.XXX.100:8443
>         fallback=127.0.0.1:80
>         real=192.168.50.4:8443 masq
>         real=192.168.50.5:8443 masq
>         scheduler=rr
>         checktype=connect
>         protocol=tcp
> 

----- Original Message ----- 
From: "Joseph Miller" <[email protected]>
To: <[email protected]>
Sent: Tuesday, January 17, 2006 11:13 AM
Subject: Re: ldirectord memory leak in Ultramonkey 3


> Glen Kendell wrote on 22/11/05 17:57:
>> I think the idea should be to figure out if what you are seeing is really a
>> memory leak or just normal memory management.  So you will need to do a
>> little testing to help out in that regard.
>> 
>> Add the -d switch to ldirectord to get debugging output.  Stop heartbeat on
>> the director (or both if you are running two) and then launch ldirectord
>> interactively on one director so you can see the output.
> 
> Finally got a test environment set up. Unfortunately, we don't seem to be able to reproduce the 
> problem on the test servers - they work perfectly.. so we swapped out one of the test servers for a 
> production one, and the production one is still exhibiting the leak.
> 
> It appears that the leak just occurs over time - seems to be unaffected regardless of whether 
> ldirectord is routing traffic or not.. the software set up is the same, which makes us wonder if 
> maybe some sort of hardware issue at play...
> 
> Anyway, below are links to a debug log of 1) the production server which is showing the leak, and 2) 
> a test server (set up the same) which is not. Personally I can't see much that's different, but a 
> more trained eye might spot something.
> 
> 1) http://asuka.googlehax.com/ldirectord-leaking-stderr.txt
> 2) http://asuka.googlehax.com/ldirectord-noleak-stderr.txt
> 
> 
>> 
>> On 11/21/05 1:57 PM, "Joseph Miller" <[email protected]> wrote:
>> 
>>> Hi Glen,
>>>
>>> Glen Kendell wrote:
>>>> Out of curiosity, are you seeing anything in your log or debug log files?
>>> I can't see anything unusual in the log files.. how do you turn on debug
>>> logging?
>>>
>>>> What versions of the other UM packages do you have in place?
>>> director2:~# uname -a
>>> Linux director2 2.6.8-2-686 #1 Thu May 19 17:53:30 JST 2005 i686 GNU/Linux
>>>
>>> Version numbers from dpkg -p <package name>
>>> ultramonkey 3-1um.1
>>> heartbeat 1.2.3.cvs.20050927-1bpo1
>>> ldirectord 1.2.3.cvs.20050927-1bpo1
>>> iproute  20041019-3
>>> ipvsadm  1.24+1.21-1
>>>
>>>
> 
> 
> -- 
> Thanks,
> Joseph Miller
> Application Developer
> _________________________________________
> http://www.securepaytech.com/
> Secure Payment Technologies
> Online Credit Card Transaction Processing
> Phone: (64 3) 961 9555
> Email: [email protected]
> 





-- 
Ultra Monkey - http://www.ultramonkey.org/
To UNSUBSCRIBE, email to [email protected], with a body:
unsubscribe ultramonkey-users [email protected]
where "[email protected]" is YOUR email address.