Re: ldirectord memory leak in Ultramonkey 3

I think the idea should be to figure out if what you are seeing is really a
memory leak or just normal memory management.  So you will need to do a
little testing to help out in that regard.

Add the -d switch to ldirectord to get debugging output.  Stop heartbeat on
the director (or both if you are running two) and then launch ldirectord
interactively on one director so you can see the output.

Simluate an external load on your services.  In separate console windows,
keep track of memory usage using top and free -m.

If there truly is a memory leak, you will see the process grow unabated
until it forces swapping.  You will see how much swapping you have with
free.  If instead you see the cached memory plateau and no swapping you are
witnessing typical Linux memory management.  Cached memory is essentially
free, it is just keeping it in cache in case the same data is needed again.

The reason Horms asked about your perl version is because there have been
memory leaks in the past that were from shared libs like ssleay, not
ldirectord per se. 

Since you have a machine named director2, is it correct to assume you have 2
directors for high availability?  Please post your heartbeat config and
haresources for fun.

Repeat this process with your second director.  Or introduce a new test
machine with a fresh install of Sarge and UM 3 to see if you can reproduce
it.

Hopefully you will have an opportunity to do some debugging over the holiday
weekend without disruption of your service.

Good luck and I look forward to seeing what you come up with.

- Glen

On 11/21/05 1:57 PM, "Joseph Miller" <[email protected]> wrote:

> Hi Glen,
> 
> Glen Kendell wrote:
>> 
>> Out of curiosity, are you seeing anything in your log or debug log files?
> 
> I can't see anything unusual in the log files.. how do you turn on debug
> logging?
> 
>> What versions of the other UM packages do you have in place?
> 
> director2:~# uname -a
> Linux director2 2.6.8-2-686 #1 Thu May 19 17:53:30 JST 2005 i686 GNU/Linux
> 
> Version numbers from dpkg -p <package name>
> ultramonkey 3-1um.1
> heartbeat 1.2.3.cvs.20050927-1bpo1
> ldirectord 1.2.3.cvs.20050927-1bpo1
> iproute  20041019-3
> ipvsadm  1.24+1.21-1
> 
> 
>> And what kind
>> of load is being placed against the services?  Sorry if I missed that
>> information if you already posted earlier..
> 
> Well, over a (rougly) 24 hour period, we have approx 7500 connections, 61,000
> incoming packets,
> 58,000 outgoing (according to ipvsadm -L --stats)
> 
> We expect to see far more traffic than this fairly soon :-)
> 





-- 
Ultra Monkey - http://www.ultramonkey.org/
To UNSUBSCRIBE, email to [email protected], with a body:
unsubscribe ultramonkey-users [email protected]
where "[email protected]" is YOUR email address.