Re: implementing a set of queue-processing servers

Stephen Adkins writes:
> QUICK AND DIRTY SINGLE-SERVER SOLUTION
> 
> I implemented a quick-and-dirty single-server solution, where
> I use a single server to process requests.  I simply poll the
> request table in the database once a minute for new requests,
> and if they exist, I process them.
> 
> Now I am looking to upgrade this for higher throughput (multiple
> parallel servers), lower background load (no polling during quiet
> periods), and lower latency (immediate response to queue insertion
> rather than waiting for the next poll interval).

I like this solution.  Are you finding performance problems?  One
thing is to execute "process_queue" right after doing the insert.
Remember that databases are great at assuring atomicity and
persistency.

Apache/mod_perl is the best all around application server available.
What's simpler than LWP::Request with an URL and a return of a Perl
(or XML if it's another language) data structure?  Run eval
(or XML::Parser) and off you go.  You can wrap this, but how many
message types do you have?

Rob