Re: [PATCH] improved store-file

Nisan Bloch wrote:

> Hi
> 
> At 11:57 AM 2004/10/29, Alexander Malysh wrote:
>>Hi Kalle,
>>
>>some comments to your patch...
>>
>>1) don't use dict if you don't know how many items will be stored (dict
>>filled to more as 80% works very slow). Here we should use something like
>>red-black-tree.
> 
> 
> Or just change the hash function the dict code uses.

nope, hash function has nothing todo with this issue. Hashtables generally
doesn't work well when filled above 80%. So you have only one option:
dynamicaly size change (that is very very costly)

> I have attached some code for a different hash key.
> Under testing with 200,000 items in the dict. Test inserts, gets and then
> deletes all the items.
> uuid keys: new hash +-9secs; current 75secs
> int keys: new hash +-5secs;  current +-11mins

which algo are you using here?

> 
> 
> I can pass on the dict patches to implement this.
> 
> Nisan
>>2) don't call any functions in signal handler that are not signal safe
>>(you call gwthread_wakeup_all())
>>3) check for 'file != NULL' in 'int store_save(Msg *msg)'. Because file !=
>>NULL is our marker for enabled store-file support (store-file support is
>>optional).
>>
>>
>>Kalle Marjola wrote:
>>
>> > Attached a patch (against CVS) that greatly improves store-file:
>> > It uses Dict instead of List to save messages to memory.
>> > Moreover, acknowledged messages are immediately cleaned from
>> > memory, instead of being done in a background thread (that thread
>> > simple dumps current state to file to reduce startup-times).
>> > And, above this all, now it is possible to define the frequency
>> > how often this 'cleanup' is done..
>> >
>> > Thus it is far more powerful if queues are created, and I
>> > greatly encourage all kannel instances to use it, at least
>> > after it has gained adequate extra testing (I have done some
>> > tests but more are naturally needed)
>> >
>> > As part of the patch, smskannel.conf is updated to enable
>> > store-file - I know that lots of Kannel installation are based
>> > on smskannel.conf which is then modified...
>> >
>> > Comments?
>> > I can put this into CVS immediately after Stipe puts out 1.4.0,..
>> >
>>
>>--
>>Thanks,
>>Alex
> 
> 
> --------------------------------
> www.clickatell.com
> Any message, anywhere
> Phone: +27 21 9487150

-- 
Thanks,
Alex