HTTP retries and queue delay

Hi

I'm running a Kannel server that since a couple of days is getting a big 
volume of traffic (around 20K daily). I use it with an HTTP service to 
route all incoming messages to a PHP application with a mysql database 
and so.

This morning, the PHP server got overloaded and could not process about 
2500 requests in an hour, so those requests got queued on the smsbox 
http queue. When the server lowered its stress, kannel started retrying 
all of that queued requests. This is an extract of the smsbox log, 
showing that retries are being performed:

2005-07-07 10:27:09 [18321] [6] DEBUG: HTTP: Queue contains 2538 
outstanding requests
2005-07-07 10:27:39 [18321] [6] DEBUG: HTTP: Queue contains 2537 
outstanding requests
2005-07-07 10:28:09 [18321] [6] DEBUG: HTTP: Queue contains 2536 
outstanding requests
2005-07-07 10:28:39 [18321] [6] DEBUG: HTTP: Queue contains 2535 
outstanding requests
2005-07-07 10:29:09 [18321] [6] DEBUG: HTTP: Queue contains 2534 
outstanding requests
2005-07-07 10:29:39 [18321] [6] DEBUG: HTTP: Queue contains 2533 
outstanding requests
2005-07-07 10:30:09 [18321] [6] DEBUG: HTTP: Queue contains 2532 
outstanding requests
2005-07-07 10:30:39 [18321] [6] DEBUG: HTTP: Queue contains 2531 
outstanding requests
2005-07-07 10:31:09 [18321] [6] DEBUG: HTTP: Queue contains 2530 
outstanding requests
2005-07-07 10:31:39 [18321] [6] DEBUG: HTTP: Queue contains 2529 
outstanding requests

But, please take care on the date: only a single HTTP request is being 
retried every 30 seconds!! That's due to the http-queue-delay param that 
was set to 30 sec. A quick calculation tells me that it will last about 
20 hours to empty the queue. Digging into the smsbox.c code, I have 
found this schema:

1033    static void http_queue_thread (void) {
    ...
1044        while ((id = gwlist_consume(smsbox_http_requests)) != NULL) {
1045         /*
1046          * Sleep for a while in order not to block other operting 
requests.
1047          * Defaults to 10 sec. if not given via http-queue-delay 
directive in
1048          * smsbox group.
1049          */
1050         if (http_queue_delay > 0)
1051             gwthread_sleep(http_queue_delay);

So it means that even if a retried message is successfully delivered in, 
let's say, 1 second, the next thread will effectively wait 30 seconds to 
process another message. Or, by default, 10 seconds.


Questions:

1 - If I modify the http_queue_delay on the kannel.conf, will the 
outstanding HTTP queue be flushed, and so, will I loose the messages?

2 - Wouldn't it be better if we try to compose another schema to avoid 
this one, and sleep only between retry runs?

Thanks a lot and keep with the good work !!

Juan Fernández