Huge Cricket Installations

Hi,

 

I’ve a Cricket install (1.0.5) that polls over XXXX nodes. Obviously this is too many for a single box to poll, so I’ve spread the load over three machines. The actual Cricket installation is in a directory on a NetApp NAS that is mounted on each machine over NFS. This in itself is problematic because of locking problems and so on but for the most part it works pretty well and facilitates a quick turn-around in the event that a machine needs to be replaced.

 

I’ve created multiple subtree sets in order to reduce the time taken to complete a polling cycle. We use Cricket to report on our CPE and new CPE are being added to the network all of the time so manual management of the subtree sets would be very time consuming. Some of the subtree sets are actually automatically generated out of a database backend (I’ve a script that automagically ‘discovers’ every node on the network, identifies what it is and adds it to the appropriate table in the DB) – the sets are treated like buckets: as soon as a set has 700 nodes in it, we start to fill another set. These sets are spread across the three polling machines.

 

Generally this setup works well for us but there are number of issues that we are experiencing. Firstly, the load on the boxes is a constant problem. I’ve tried to optimise things so that each subtree set takes less than ten minutes to complete. This is fine except smaller subtree sets means more collection process which means higher memory usage and higher load. This is particularly problematic on the machine that runs the UI since it is very noticeable to users that the machine is under such a heavy load.

 

Compiling the Cricket config often takes an absolute age and consumes huge amounts of memory (at the moment I’ve a compile that has been running for over 15 minutes and still isn’t complete; it’s consuming 3798M of memory, 700 of which is resident). More often than not the compile bombs out with a segmentation fault (11) and compiling is a hit and miss affair regardless of which of the three machines I run the compile on. The resulting config.db is over 100M in size.

 

The machines are all 64-bit Dells (two are PowerEdge 1850s, two are 1950s) with 1 GB RAM, 5GB swap, running FreeBSD 6.1-STABLE.

 

Does anyone on the list have experience with large Cricket installations? If so, have you any pointers or tips that you’d like to share?

 

Dermot Williams

Senior Network Engineer

Irish Broadband Internet Services Ltd.

 

+353 1 4818481

+353 86 3887961

 


-- 
No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.430 / Virus Database: 268.14.11/543 - Release Date: 20/11/2006 21:20
 
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
cricket-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/cricket-users