load problems with Majordomo and bulk_mailer

I have a number of moderately large lists (300-3000) whose processing is 
causing CPU loading problems on my majordomo server.  I've had some 
success with a few things I've tried, but I'd appreciate your 
suggestions for a better way to throttle the load.

Here's what I've done so far:

With the original vanilla majordomo install:
1) these large lists were hanging and being delayed by bad addresses 
(over which I have no control)
2) I was seeing extended 100%CPU utilizations
3) I was seeing miserable throughput, which I was attributing to 
n-squared proportionality
4) I was concerned with sending envelopes with some 30-50 messages to 
ISPs like AOL.


Then I installed bulk_mailer and set maxrcpts=10 and cured some of these 
problems:
1) a bad address just hangs the group of 10 mailings it's in
2) I ended up with even heavier....but much briefer...loads
3) throughput has turned into an avalanche...which was a problem
I was now seeing uptime load averages as high as 200 or more and I was 
getting errors from delayed or aborted operations (Draining Input, 
Operating System error, etc), some duplicate mailings.

So I uncommented the section in majordomo.cf that defers incoming mail 
to majordomo when the uptime load average gets above a defined value and 
set that to 10.

That helped a bit...but not a lot...here's why:
a single job gets submitted into majordomo....which processes it and (so 
far no great load, so no throttling) hands it off to bulkmailer which 
splits it into multiple jobs of 10 recipients each.../this/ puts a 
substantial load on the majordomo host, but the load is already past the 
gatekeeper.

What's happening is that numerous jobs get accepted by majordomo 
/BEFORE/ the load average gets high enough to defer further mailings. 
The numerous jobs that did get accepted by majordomo then get expanded 
by bulk-mailer and the system gets clobbered.
So majordomo throttling will not keep the CPU from going to 100%, but it 
will defer new jobs once it has reached the load average 
trigger...somewhat too late.  Though the uptime load average trigger is 
set at 10, once bulk_mailer expands the jobs and things get rolling, the 
load average can go as high as 30....even though majordomo deferred 
things after it hit 10!

Does anyone know of a better way to throttle majordomo and/or bulk_mailer?
-- 
====
Stewart Dean, Unix System Admin, Henderson Computer Resources
Center of Bard College, Annandale-on-Hudson, New York  12504
[email protected]  voice: 845-758-7475, fax: 845-758-7035