Re: Kannel HTTP MO delivery failing under load

Hi Giulio,

Giulio Harding wrote:

> The disparity between the two sides of the TCP connections is  probably 
> due to backlog - only 5 TCP connections are ever established  with the 
> HTTP server at a time, but the rest of the connections from  Kannel are 
> 'pending' as per OS/apache TCP backlog limits. These  connections won't 
> receive a 'connection refused', they'll simply wait  until resources are 
> available, whereupon they'll be fully established.
> 
> Because these connections aren't explicitly failing, and because of  the 
> rate of incoming MO messages, smsbox appears to be simply  creating HTTP 
> connections faster than they are being used. The only  solution is to 
> forcibly reduce the rate of message delivery, or to  limit the number of 
> HTTP connections to the application HTTP server  (as mentioned by 
> Alexander Malysh in his reply to my original email).
> 
> Also, those 'strange' sockets that Kannel has look like they've been  
> closed on the other side (by the application HTTP server) and are  thus 
> useless, and simply haven't been cleaned up. When a HTTP  connection is 
> closed by the server, it should be cleaned up on  Kannel's side as soon 
> as possible. They should really be cleaned up  before attempting to 
> create new HTTP connections.

I agree in this analysis.

> I think this is the crux of the problem: this TODO, and our issue  would 
> be addressed by the patch that Alexander has offered - given  that he's 
> indicated that it is a fairly simple patch, and the issue  seems (to me) 
> quite serious, I think this is worth focussing on ASAP.

agreeing too.

>>> These look like they may be directly related to the problem we're  
>>> experiencing - is anyone working on these tasks, and if so, is  there 
>>> an ETA to implementation?
>>
>>
>> can you please retry to leaverage the MO load with let's say 2-4  
>> smsbox connections....
>>
>> It's still curious why sockets are keept open that way...
> 
> 
> Ok, I'll try this out, with multiple smsboxes on the same machine,  and 
> possibly multiple smsboxes on multiple machines (if I can source  some) 
> - I'll also try fiddling with the priority of the bearerbox  process 
> (using nice) to see if I can get the SMPP and HTTP processing  better 
> balanced...
> 
> However, none of these workarounds will guarantee correct behaviour  
> under load, as there is nothing preventing Kannel from opening too  many 
> sockets. As I mentioned before, I think this is a bug.

yep, can you please add this to a full but report to http://bugs.kannel.org/, in 
order we have a correct track of the issue? Thanks in advance.

Stipe

-------------------------------------------------------------------
Kölner Landstrasse 419
40589 Düsseldorf, NRW, Germany

tolj.org system architecture      Kannel Software Foundation (KSF)
http://www.tolj.org/              http://www.kannel.org/

mailto:st_{at}_tolj.org           mailto:stolj_{at}_kannel.org
-------------------------------------------------------------------