Re: [FYI] New 'dlr-storage = spool' available in SVN trunk

Hi Stipe,

dlr_clean sounds good - yes we have a few old ones lying around,
especially annoying with internal storage as the seek times only get
worse as it grows. That would be a very nice addition from a
reconciliation perspective ... only issue is we'd need a custom status,
no? i.e. UNKNOWN? 

One thing I was going to ask (still running basic testing) is that the
DLR spool files aren't easily human readable - to run a scripted manual
clean-up so - but the dlr_clean would resolve the need to 'read' these.
Was thinking about looking into a util like Lars's test_msg - probably
still useful.

Also does enabling VERIFIED work? I've run from source dir as dlr_spool
directory (i.e. dlr-spool = . ) but I end up with 1800+ in status.
Start-up is only adding 7s (from logs) so I thought I'd check the hit on
verifying .. (would normally be in a clean directory).

a gdb backtrace gives me ..

#0  0x00000000 in ?? ()
#1  0x080650df in verified_file (filename=0x81c0fa8
"./debian/kannel.default", sb=<optimized out>, 
    tflag=<optimized out>, ftwbuf=<optimized out>) at gw/dlr_spool.c:180
#2  verified_file (filename=0x81c0fa8 "./debian/kannel.default",
sb=0xbfffeba0, tflag=0, ftwbuf=0xbfffeee0)
    at gw/dlr_spool.c:161
#3  0x005cf2e7 in process_entry (data=0xbfffeecc, dir=0xbfffec74,
name=<optimized out>, namlen=14, d_type=8)
    at ftw.c:470
#4  0x005cf631 in ftw_dir (data=0xbfffeecc, st=0xbfffecd0,
old_dir=0xbfffeda4) at ftw.c:546
#5  0x005cf1b7 in process_entry (data=0xbfffeecc, dir=0xbfffeda4,
name=<optimized out>, namlen=6, d_type=4)
    at ftw.c:467
#6  0x005cf631 in ftw_dir (data=0xbfffeecc, st=0xbfffee6c, old_dir=0x0)
at ftw.c:546
#7  0x005cff62 in ftw_startup (dir=<optimized out>, is_nftw=-1208145600,
func=0xffffffc8, descriptors=20, 
    flags=1) at ftw.c:772
#8  0x005d00a7 in __new_nftw64 (path=0x81bf8f0 ".", func=0x8065090
<verified_file>, descriptors=20, flags=1)
    at ftw.c:856
#9  0x08065182 in for_each_file (dir_s=<optimized out>, cb=0x8065090
<verified_file>, 
    ignore_err=<optimized out>) at gw/dlr_spool.c:252
#10 0x0806523d in dlr_init_spool (cfg=0x81b4fd0) at gw/dlr_spool.c:626
#11 0x08061a59 in dlr_init (cfg=0x81b4fd0) at gw/dlr.c:252
#12 0x08053155 in main (argc=2, argv=0xbffff164) at gw/bearerbox.c:706

I'm still trying to get my head around how the unpack func is
set-up ,etc.

Cheers,
Alan



On Thu, 2012-08-09 at 13:47 +0200, Stipe Tolj wrote:
> Am 07.08.2012 23:43, schrieb Alan McNatty:
> > Hi Stipe,
> >
> > Sounds great - as you say being able to use disk based replication
> > techniques would be good but also providing persistence as a default
> > option is also very valuable. I'll be keen to do some tests on IO
> > performance - my biggest concern for larger installs (as often disk
> > choice is not necessarily under our control).
> >
> > If anyone cooks up any test scripts to analyse please feel free to share
> > (as I will). I'm just trying to think of an easy way to generate several
> > thousand in the DLR store and have them all expire (identical validity
> > period or similar).
> >
> > Thanks again Stipe - nice work!
> 
> Hi Alan,
> 
> thanks a lot.
> 
> In fact I'm also providing a patch to the mailing list shortly that 
> includes 2 new methods for the DLR abstraction modules, implementing the 
> functions:
> 
> - size_t dlr_purge(time_t t) : removing all entries that are older then 
> t seconds, and providing the ammount of removed entries. This can be 
> used via the HTTP admin interface to "purge" all zombie DLRs from the 
> storage. Normally people need to do this via external logic, i.e. 
> scripts calling DELETE something calls for their DB storages. Via the 
> HTTP interface making the method homogenate for all DLR storage 
> implementation is a much cleaner approach IMO.
> 
> - size_t dlr_clean(time_t) : the same logic as above, BUT in addition it 
> sends a DLR event 2 (delivery failed) DLR to the smsbox. Which means we 
> provide a callback mechanism for any smsbox connection that wants to 
> ensure the back-end ESMEs (HTTP or SMPP EMSE) get informed about the 
> "final state" that the DLR was never confirmed from the upstream SMSC 
> and we (Kannel) consider it as not delivered.
> 
> Cheers,
> Stipe
>