RE: Re: Problematic Kannel behaviour with large message stores

Hi,

yes, store-spool is ready for use in CVS.
Please report how it's going.

Mario Noboa wrote:

>  
> Hi list,
>  
> I have problems when kannel has one releases message queue. The change of
> store-spool maybe can help me.
>  
> This change is ready in the present cvs?
> 
>  
> 
> Thanks
> 
> Mario
> 
>  
> 
> ----- Original Message ----- From: "Alexander Malysh" <[EMAIL PROTECTED]>
> 
> To: <[email protected]>
> Sent: Friday, September 08, 2006 1:03 PM
> Subject: Re: Problematic Kannel behaviour with large message stores
>  
>  
> Hi,
>  
> will try todo it next week. I have it up to 80% ready with possibility to
> select store file or store spool...
>  
> Thanks,
> Alex
>  
> Vincent CHAVANIS wrote:
>  
> Alex,
>  
> Could you please review you patch to fit the actual CVS tree, as we can
> test it.
> Then, how it comes this patch does not have been commited as it was ++2
>  
> regards
>  
> Vincent
>  
> --
> Telemaque - 06200 NICE - (FR)
> Service Technique/Reseau - NOC
> Developpement SMS/MMS/Kiosques
>  <http://www.telemaque.fr/> http://www.telemaque.fr/
> [EMAIL PROTECTED]
> Tel : +33 4 93 97 71 64 (fax 68)
> ----- Original Message -----
> From: "Alexander Malysh" <[EMAIL PROTECTED]>
> To: <[email protected]>
> Sent: Friday, September 08, 2006 12:06 PM
> Subject: Re: Problematic Kannel behaviour with large message stores
>  
>  
> Hi,
>  
> you can search for my patch
>  
> (
>
<http://article.gmane.org/gmane.comp.mobile.kannel.devel/20690/match=spool>
> http://article.gmane.org/gmane.comp.mobile.kannel.devel/20690/match=spool)
> that replaces store file with store spool directory. This works with
> similar performance when you put your store dir on reiserfs or tmpfs
> under linux. It would be great to hear test results for this patch.
> Please note that patch is very old and could not apply that you should
> fix manually.
>  
> Thanks,
> Alex
>  
> Giulio Harding wrote:
>  
> We've been doing some more stress-testing of Kannel, this time under
> high-load over long periods of time, an we've encountered some
> problems when the message store grows large, of the order of hundreds
> of thousands of messages (in our case, because we're delivering MTs
> slower than we are receiving MOs, and due to the use of synchronous
> responses to MO HTTP requests, the MTs are queuing at the bearerbox).
> We understand that this is an extreme situation, and unlikely to be
> encountered in normal usage, but we want to cover all our bases
> (worst-case scenarios) and make sure that Kannel behaves correctly/
> predictably in bad situations.
>  
> - It seems that reading from the message store during Kannel startup
> is single-threaded, and CPU bound - for large stores (several hundred
> thousand messages), this can result in very long start-up times,
> since Kannel seems to not process any messages in the store until it
> has finished reading them *all* into memory from disk.
>  
> - Why can't they simply be processed from disk? (i.e. read message
> from disk, process message, read next message from disk) Given that
> the messages are in a persistent store, why do they need to be in
> memory as well?
>  
> - The bearerbox memory usage grows by around 100MB per hundred
> thousand messages in the store, as the messages are read from the
> store, which leads to VM thrashing under extreme circumstances, as
> the bearerbox footprint grows beyond the RAM size of the server...
> For one test, after the initial loading of ~634,000 MTs from the
> store, and bearerbox was using ~780M of RAM. 1+ KB per message seems
> a lot, given that messages seems to be occupying more space in memory
> than on disk (that number of MTs was consuming about 255MB on disc)
>  
> - Once messages are read into memory, they seem to be copied into
> another queue (this exacerbates the problem above) - why is this?

-- 
Thanks,
Alex