[PATCH] RE: Problem with spool store - missing sms_type

-----Original Message-----
From: Rene Kluwen [mailto:[email protected]] 
Sent: Wednesday, 11 August, 2010 13:41
To: 'Alexander Malysh'
Cc: 'Tomasz'; '[email protected]'
Subject: [PATCH] RE: Problem with spool store - missing sms_type

See attached patch.

== Rene

-----Original Message-----
From: Alexander Malysh [mailto:[email protected]] On Behalf Of Alexander Malysh
Sent: Wednesday, 11 August, 2010 09:27
To: Rene Kluwen
Cc: 'Tomasz'; [email protected]
Subject: Re: Problem with spool store - missing sms_type

Hi Rene,

not looking in the code of smppbox: msg->type is set when you create message e.g. msg_create(sms) BUT store shows not this type. Store shows msg->sms.sms_type which you have to set explicitly to mo/mt-push/dlr-mo/dlr-mt.

Thanks,
Alexander Malysh

Am 10.08.2010 um 21:49 schrieb Rene Kluwen:

> Looking into the msg.c source code (function msg_pack()) it is not even possible for smppbox to send a message with an invalid msg->type to bearerbox.
> 
> I wonder what might be wrong.
> 
> == Rene
> 
> -----Original Message-----
> From: Tomasz [mailto:[email protected]]
> Sent: Tuesday, 10 August, 2010 20:22
> To: Rene Kluwen
> Subject: Re: Problem with spool store - missing sms_type
> 
> When I call http://domain.pl:13000/store-status I can see the table of 
> all queued messages (if any). And all those messages which were 
> submitted via openSMPPBOX have "Type" field empty there. However all 
> messages submitted by SMSBOX have this value filled correctly.
> 
> Tomasz
> 
> 
> 
> W Twoim liście datowanym 10 sierpnia 2010 (19:06:21) można przeczytać:
> 
>> Where exactly in the http admin page do you see that msg_type is empty?
> 
>> -----Original Message-----
>> From: [email protected] [mailto:[email protected]] On 
>> Behalf Of Tomasz
>> Sent: Tuesday, 10 August, 2010 18:21
>> To: [email protected]
>> Subject: Re: Problem with spool store - missing sms_type
> 
>> Hi,
> 
>> I don't know for sure if this is openSMPPBOX issue or not but if 
>> messages are submitted via openSMPPBOX the msg_type is empty and this 
>> makes that Bearerbox crashes during restart when we have some 
>> messages queued in the spool. When submitting messages by SMSBOX (CGI 
>> push), the problem didn't exists - msg_type is set correctly.
> 
>> I can check if msg_type exists or not using http admin store-status 
>> command (when there are some queued messages). Messages submitted via 
>> openSMPPBOX have empty fields in "Type" column.
> 
>> Rene, I can provide more details with the issue, but I can't see in 
>> logs any revelant information - only PANICs during start of 
>> Bearerbox. Only at http admin page I can see that msg_type is empty.
>> But if you need please let me know what information would be helpful.
> 
>> Tomasz
> 
> 
> 
>> W Twoim liście datowanym 10 sierpnia 2010 (17:01:52) można przeczytać:
> 
>>> In the smppbox code, I don’t see anywhere where a msg is created without msg_type.
> 
>>> We use the msg_create() function and dlr_find functions to create messages.
> 
>>> 
> 
>>> If this is an smppbox issue, I would like to get more information about it.
> 
>>> 
> 
>>> == Rene
> 
>>> 
> 
>>> From: Alejandro Guerrieri [mailto:[email protected]]
>>> Sent: Monday, 09 August, 2010 23:27
>>> To: Rene Kluwen
>>> Cc: Nikos Balkanas; [email protected]
>>> Subject: Re: Problem with spool store - missing sms_type
> 
>>> 
> 
>>> Exactly.
> 
>>> 
> 
>>> The point is: during normal operation, kannel of course it doesn't 
>>> panic and will accept messages without a valid sms type. However, 
>>> they're kept on the store with an invalid format, so if you shutdown 
>>> the service with messages pending on the store, and just one of them 
>>> happens to have an invalid sms type, the service panics at startup.
>>> This is less than desirable of course, specially when you have a ton 
>>> of completely valid messages and just a bunch of invalid.
> 
>>> 
> 
>>> IMHO, kannel should reject messages with invalid sms type during 
>>> regular operation (with a WARN logged). It _shouldn't_ try to fix 
>>> them. That would take care of the problem in a "proper" way.
> 
>>> 
> 
>>> Apart from that, a way to discard invalid messages at bootup without 
>>> panicking would also be desirable
> 
>>> 
> 
>>> Regards,
> 
>>> 
> 
>>> Alex
> 
>>> On Mon, Aug 9, 2010 at 11:11 PM, Rene Kluwen <[email protected]> wrote:
> 
>>> Yes, open smppbox should correctly fill in the correct type. If it doesn't it is an error.
> 
>>> But at the same time: If one particular message has an incorrent 
>>> sms_type. Why panic? It can just discard the message and go on with normal operation.
> 
>>> == Rene
> 
> 
>>> -----Original Message-----
>>> From: [email protected] [mailto:[email protected]] On 
>>> Behalf Of Nikos Balkanas
>>> Sent: Monday, 09 August, 2010 22:34
>>> To: Alejandro Guerrieri
>>> Cc: [email protected]
>>> Subject: Re: Problem with spool store - missing sms_type
> 
>>> Hi,
> 
>>> The behaviour in store is the only correct one. sms_type could be an 
>>> MO (0), MT (2) or DLR (3). Different logic and routing is applied in each case.
>>> During startup it doesn't know which one is and correctly panics. 
>>> During operation, maybe bb can tell more, but I am not sure it is 
>>> always safe to do so. It has to discriminate between an MT, a 
>>> reroute_dlr (report_mt) and an mt_reply (from an MO). Or between an 
>>> MO and a report_mo. Anyway, it should at least be consistent, and it 
>>> should check for sms_type and if missing and absolutely sure it knows what it is, fill it in, else discard with an error.
> 
>>> This is an opensmppbox issue. It should set the correct sms_type 
>>> according to gw/msg.h.
> 
>>> BR,
>>> Nikos
>>> ----- Original Message -----
>>> From: Alejandro Guerrieri
>>> To: Nikos Balkanas
>>> Cc: Tomasz ; [email protected]
>>> Sent: Monday, August 09, 2010 9:12 PM
>>> Subject: Re: Problem with spool store - missing sms_type
> 
> 
>>> Yep, smsbox doesn't. Sqlbox, if you're not careful, does.
> 
> 
>>> The problem is with the way messages are checked. When messages are 
>>> received from a box, they go to memory first _and_ the store later. 
>>> In that case, bearerbox doesn't perform any sanity checks on the sms type field.
> 
> 
>>> Now, when messages are retrieved from the store during boot, a 
>>> sanity check is performed and the whole system panics if it 
>>> encounter a single invalid message.
> 
> 
>>> I think two things would be needed here:
> 
> 
>>> 1. Perform the same sanity checks when getting messages from boxes 
>>> and reject anything that would cause a problem when retrieved from the store.
> 
> 
>>> 2. Add an option to boot kannel discarding those corrupted messages. 
>>> A few hundred corrupted messages in the store could mean a nightmare 
>>> when trying to restart a crashed server.
> 
> 
>>> Regards,
> 
> 
>>> Alex
> 
> 
>>> 2010/8/9 Nikos Balkanas <[email protected]>
> 
>>> Hi,
> 
>>> I can verify to the thousands of kannel users all over the wold, 
>>> that smsbox doesn't have any such issue. However this seems an issue 
>>> with bearerbox as well. SMPPbox should really generate correct Msg 
>>> *, and bearerbox shouldn't pnick about them. I mean if it is happy 
>>> processing them live, why should it panic at start?
> 
> 
>>> BR,
>>> Nikos
>>> ----- Original Message ----- From: "Tomasz" <[email protected]>
>>> To: <[email protected]>
> 
>>> Sent: Monday, August 09, 2010 8:14 PM
> 
>>> Subject: Re: Problem with spool store - missing sms_type
> 
> 
>>> Hi,
> 
>>> Open SMPPBOX haven't its own queue - I submit messages to Bearerbox 
>>> via open SMPPBOX from other system. But sometimes these messages are 
>>> being queued by Bearerbox in spool.
> 
>>> But when Bearerbox is restarted while at spool there are some 
>>> messages, it PANICs and won't run.
> 
>>> The problem is because messages at spool haven't Type field. They 
>>> have SMS ID, Time, Sender, Receiver, SMSC ID, BOX ID, Β UDH, Message 
>>> fields but Type field is empty.
> 
>>> Bearerbox during start informs about it:
> 
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: Not handled sms_type within store!
> 
>>> I didn't tried submitting messages to BEARERBOX from a standard 
>>> SMSBOX yet, only by open SMPPBOX so I don't know at the moment if 
>>> this problem happens only when using open SMPPBOX.
> 
>>> @Nikos Sorry for adressing you private, it was my mistake.
> 
>>> Tomasz
> 
> 
>>> Please address list.
> 
> 
> 
>>> I didn't know that opensmppbox has now a queue. Clearly you 
>>> shouldn't have overlapping spools between bb and openssmppbox. 
>>> Configure different spool areas for each one.
> 
> 
> 
>>> BR,
>>> Nikos
>>> ----- Original Message ----- From: "Tomasz" <[email protected]>
>>> To: "Nikos Balkanas" <[email protected]>
>>> Sent: Monday, August 09, 2010 7:55 PM
>>> Subject: Re: Problem with spool store - missing sms_type
> 
> 
> 
> 
>>> Hi,
> 
> 
> 
>>> Yes, I know that they are corrupted, but all msgs in spool are always
>>> corrupted   I removed them, but all new messages queued at spool are
>>> corrupted.
> 
> 
> 
>>> They are probably incorrectly saved by Bearerbox/openSMPPBOX.
> 
> 
> 
>>> The problem starts when I want to restart Bearerbox - it displays 
>>> PANICs and won't start until I remove spool manually. It causes that 
>>> I can't restart Bearerbox if there is some queue in spool...
> 
> 
> 
>>> Tomasz
> 
> 
> 
> 
> 
>>> W Twoim liΞ’ cie datowanym 9 sierpnia 2010 (18:34:44) moΞž na przeczytaΞžΒ¶:
> 
> 
> 
> 
>>> Hi,
> 
> 
> 
>>> You have a corrupted SMS in your spool. Remove it and you will be fine.
> 
> 
> 
>>> BR,
>>> Nikos
>>> ----- Original Message ----- From: "Tomasz" <[email protected]>
>>> To: <[email protected]>
>>> Sent: Monday, August 09, 2010 7:30 PM
>>> Subject: Problem with spool store - missing sms_type
> 
> 
> 
> 
>>> Hi,
> 
> 
> 
>>> Today I've found some critical error with kannel spool store-type.
>>> When I have messages in a queue (spool) and restart Bearerbox I get
>>> Panic:
> 
> 
> 
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: Not handled sms_type within store!
>>> 2010-08-09 17:49:55 [29887] [0] PANIC:
>>> /usr/local/sbin/bearerbox(gw_panic+0x14b) [0x487f5b]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: /usr/local/sbin/bearerbox 
>>> [0x419721]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: /usr/local/sbin/bearerbox 
>>> [0x419144]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: /usr/local/sbin/bearerbox 
>>> [0x419166]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: /usr/local/sbin/bearerbox 
>>> [0x419689]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC:
>>> /usr/local/sbin/bearerbox(main+0x80f)
>>> [0x40f22f]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC:
>>> /lib/libc.so.6(__libc_start_main+0xe6) [0x7f5cdfd3b1a6]
>>> 2010-08-09 17:49:55 [29887] [0] PANIC: /usr/local/sbin/bearerbox 
>>> [0x40db09]
> 
> 
> 
>>> When I checked store-status (via http admin) I could see that "Type" 
>>> field of all messages was empty. All messages were submitted to 
>>> Bearerbox via open SMPPBOX.
> 
> 
> 
>>> My Kannel version is from latest SVN (Rev. 4837).
> 
> 
> 
> 
>