Re: PATCH: addressing an issue in smsc_at that makes the modem stop reporting incoming SMSes

I'm glad to hear that my contribution to Kannel proved useful.
That was the least I could do - having Kannel being so useful to me over 
the years now.

Soon I'll propose a much better way to handle on-SIM messages (the 
'sim-buffering' use case).

Best regards,
Andrija

Alexander Malysh wrote:
> Hi,
>
> it's OK now and I applied your patch to cvs. Just stresstested it and it
> seems to fix errors like this:
> CMS ERROR: Unknown error. -> maybe Sim storage is full? I'll have a look at
> it. (500)
>
> Thanks!
>
> Andrija Petrovic wrote:
>
>   
>> Let's try again...
>>
>> Best regards,
>> Andrija
>>
>> Alexander Malysh wrote:
>>     
>>> Hi,
>>>
>>> it doesn't apply cleanly to current CVS. Could you please rediff?
>>>
>>> Andrija Petrovic wrote:
>>>
>>>   
>>>       
>>>> I am sorry for messing up with the tabs-spaces issue.
>>>> The file should be correct now.
>>>>
>>>>  From Andreas Fink I have learned that there are problems in reading the
>>>> patch when I attach it as an ASCII file. So, I'm sending it gzipped.
>>>>
>>>> Best regards,
>>>> Andrija
>>>>
>>>> Alexander Malysh wrote:
>>>>     
>>>>         
>>>>> Hi,
>>>>>
>>>>> patch is OK and I'm willing to apply it BUT: you totally screwed
>>>>> indentation. Please use only spaces and not tabs. If you or any other
>>>>> fix indentation I will apply this patch to CVS.
>>>>>
>>>>> Thanks again for your patch.
>>>>>
>>>>> Andrija Petrovic wrote:
>>>>>
>>>>>   
>>>>>       
>>>>>           
>>>>>> OK,
>>>>>>
>>>>>> here's the patch with original text. Once we fix this modem issue,
>>>>>> I'll propose another patch that will speed up the process of reading
>>>>>> SMSes from the memory (AT+CMGL instead of AT+CMGR).
>>>>>> Anyhow, I'm eager to provide my help/collaboration to make smsc_at
>>>>>> work in a stable manner.
>>>>>>
>>>>>> ================================================
>>>>>>
>>>>>> The main goal of the patch is to follow a simple rule:
>>>>>> "If the modem reports an error during SMS sending, don't retry the
>>>>>> sending, but ask it what it's got to say"
>>>>>>
>>>>>> So, any loops related to SMS sending are removed:
>>>>>>
>>>>>> 1. loop in at2_send_messages has been removed. Messages will be sent
>>>>>> one by one from the at2_device_thread main loop -
>>>>>> while(!privdata->shutdown) 2. loop in at2_send_one_message has been
>>>>>> removed, so the single SMS sending either succeeds or fails
>>>>>>
>>>>>> Consequences of the patch:
>>>>>>
>>>>>> 1. the define RETRY_SEND has been renamed to RETRY_WRITE (because it
>>>>>> is also used in writing a line to the modem), there is no RETRY_SEND
>>>>>> any more 2. the user needs to rely upon the sms-resend-retry mechanism
>>>>>> - this works fine AFAIK. E.g. put sms-resend-retry = 5 in the conf
>>>>>> file. Without such a line, kannel will not try to resend the SMS
>>>>>>
>>>>>> Best regards,
>>>>>> Andrija
>>>>>>
>>>>>> P.S.
>>>>>>
>>>>>> Long story short, the problem is in forcing the modem to send an SMS -
>>>>>> even if it reports an error.
>>>>>>
>>>>>> In the official Kannel version, there is a RETRY_SEND (equal to 3, in
>>>>>> smsc_at.h) define that defines how many times an SMS sending process
>>>>>> needs to be retried. It's  purpose is illogical - having a complete
>>>>>> and logical sms-resend-retry mechanism built in the Kannel logic.
>>>>>>
>>>>>> In the official Kannel version, if the software receives an error
>>>>>> response while trying to send the SMS, it simply tries again in a loop
>>>>>> (3 times, due to the value of RETRY_SEND). Such a behavior confuses
>>>>>> the GSM modem - on the first forcible SMS sending retry.
>>>>>>
>>>>>> If the software should report unsuccessful SMS sending through
>>>>>>  bb_smscconn_send_failed(privdata->conn, msg,
>>>>>> SMSCCONN_FAILED_TEMPORARILY, octstr_create("ERROR"));
>>>>>>
>>>>>> and let the modem "relax" (by asking it if there are new incoming SMS
>>>>>> messages pending) when it receives an error response from the modem, I
>>>>>> have seen that Siemens modems (TC 45, MC 39i) would never expose the
>>>>>> notorious issue.
>>>>>>
>>>>>> I have never encountered the issue in the last 15 months - since I
>>>>>> altered the logic within smsc_at.c. With the RETRY_SEND mechanism, I
>>>>>> have had the issue more than once daily (I operate the GSM modems in a
>>>>>> rather intense SMS traffic).
>>>>>>
>>>>>> Hope this comment will lead to a usable official version of smsc_at.c
>>>>>> in the future.
>>>>>>
>>>>>>
>>>>>> ===========================================
>>>>>>
>>>>>> Hillel wrote:
>>>>>>     
>>>>>>         
>>>>>>             
>>>>>>> Hi,
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Andreas is keen to install it, but wants it to be re-sent.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Please put in the subject the word PATCH as otherwise it's not clear
>>>>>>> which mail is the patch.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> thanks
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> *From:* Andreas Fink [mailto:[email protected]]
>>>>>>> *Sent:* 19 September 2007 11:48 AM
>>>>>>> *To:* Hillel
>>>>>>> *Subject:* Re: Feature request/wish for Kannel
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> ok so you want me to go through houndreds of mails to find the patch?
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> On 19.09.2007, at 10:24, Hillel wrote:
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Hi Andreas,
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> I think you should read what Andrija Petrovic wrote as it seems there
>>>>>>> are
>>>>>>>
>>>>>>> problems with both AT+CNMI operation and AT+CMGR.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Both patches seem to have been submitted but never implemented.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Rgds
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> From: Andrija Petrovic [mailto:[email protected]]
>>>>>>>
>>>>>>> Sent: 19 September 2007 10:09 AM
>>>>>>>
>>>>>>> To: [email protected] <mailto:[email protected]>
>>>>>>>
>>>>>>> Subject: Re: Feature request/wish for Kannel
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> I'd better not answer directly to your direct question. Direct answer
>>>>>>> is
>>>>>>>
>>>>>>> "this patch has nothing to do with SIM card being filled up, but
>>>>>>> applying
>>>>>>>
>>>>>>> this patch will stop the SIM card from being filled up" :).
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> A complete answer should take into account the mode of operation of
>>>>>>> the AT
>>>>>>>
>>>>>>> modem.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> 1. If the mode of operation is AT+CNMI (waiting for new message
>>>>>>>
>>>>>>> notifications), there is no reason why the modem should fill up the
>>>>>>> SIM
>>>>>>>
>>>>>>> card. Of course - with the current (unstable) version of smsc_at the
>>>>>>> SIM
>>>>>>>
>>>>>>> card used to fill up. Therefore, when I started working with Kannel,
>>>>>>> SIM
>>>>>>>
>>>>>>> overfilling was my primary concern. It turned out that once the modem
>>>>>>>
>>>>>>> started working correctly, with no "freezing" due to SMS send
>>>>>>> misbehavior,
>>>>>>>
>>>>>>> the SIM card never received a single SMS. All SMSes were consumed by
>>>>>>> the
>>>>>>>
>>>>>>> AT+CNMI mechanism (through +CMT notifications). When I say "never",
>>>>>>> that
>>>>>>>
>>>>>>> means 15 months on 3-4 modems synchronously, receiving up to 940
>>>>>>> SMSes/day,
>>>>>>>
>>>>>>> with peaks up to 30 SMSes/minute. (the figure of 30 SMSes/minute may
>>>>>>> be
>>>>>>>
>>>>>>> another topic for discussion, due to the hard limit of 10
>>>>>>> SMSes/minute for
>>>>>>>
>>>>>>> sending...)
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> 2. If the mode of operation is AT+CMGR (reading the SMSes that
>>>>>>> reached the
>>>>>>>
>>>>>>> SIM card), my actual patch did not address this problem. This part of
>>>>>>>
>>>>>>> smsc_at will be a matter of my next patch. Mr Andreas Fink did not
>>>>>>> take
>>>>>>>
>>>>>>> advantage of the AT+CMGL command (listing all the SMSes in the SIM
>>>>>>> card).
>>>>>>>
>>>>>>> That command speeds up the process of reading the SMSes from the SIM
>>>>>>> card by
>>>>>>>
>>>>>>> at least an order of magnitude. In May of 2006 I submitted a rather
>>>>>>> large
>>>>>>>
>>>>>>> patch that solved this problem, but the patch never made it to
>>>>>>> submission.
>>>>>>>
>>>>>>> So, if you ask me, do not use the sim-buffering option for now.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Andrija Petrovic
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> P.S. Just as a curiosity - what do you use Kannel for? What is the
>>>>>>>
>>>>>>> exploitation scenario? Please, do not answer if it is a business
>>>>>>> secret or
>>>>>>>
>>>>>>> any type of confidential information.
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Hillel wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Will your patch fix the problem with the SIM card filling up and not
>>>>>>> being
>>>>>>>
>>>>>>> cleared out and so new SMSs not being able to come in?
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Andreas Fink
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> Fink Consulting GmbH
>>>>>>>
>>>>>>> Global Networks Schweiz AG
>>>>>>>
>>>>>>> BebbiCell AG
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>> ---------------------------------------------------------------
>>>>>>>
>>>>>>> Tel: +41-61-6666330 Fax: +41-61-6666331  Mobile: +41-79-2457333
>>>>>>>
>>>>>>> Address: Clarastrasse 3, 4058 Basel, Switzerland
>>>>>>>
>>>>>>> E-Mail:  [email protected] <mailto:[email protected]>
>>>>>>>
>>>>>>> www.finkconsulting.com www.global-networks.ch www.bebbicell
>>>>>>> <http://www.bebbicell>.ch
>>>>>>>
>>>>>>> ---------------------------------------------------------------
>>>>>>>
>>>>>>> ICQ: 8239353 MSN: [email protected] <mailto:[email protected]> AIM: smsrelay
>>>>>>> Skype: andreasfink
>>>>>>>
>>>>>>> Yahoo: finkconsulting SMS: +41792457333
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>  
>>>>>>>
>>>>>>>       
>>>>>>>           
>>>>>>>               
>>>>>       
>>>>>           
>>>       
>
>