PATCH: addressing an issue in smsc_at that makes the modem stop reporting incoming SMSes

Andrija Petrovic <[email protected]>
Newsgroups gmane.comp.mobile.kannel.devel
Message-ID <[email protected]>
OK,

here's the patch with original text. Once we fix this modem issue, I'll 
propose another patch that will speed up the process of reading SMSes 
from the memory (AT+CMGL instead of AT+CMGR).
Anyhow, I'm eager to provide my help/collaboration to make smsc_at work 
in a stable manner.

================================================

The main goal of the patch is to follow a simple rule:
"If the modem reports an error during SMS sending, don't retry the 
sending, but ask it what it's got to say"

So, any loops related to SMS sending are removed:

1. loop in at2_send_messages has been removed. Messages will be sent one 
by one from the at2_device_thread main loop - while(!privdata->shutdown)
2. loop in at2_send_one_message has been removed, so the single SMS 
sending either succeeds or fails

Consequences of the patch:

1. the define RETRY_SEND has been renamed to RETRY_WRITE (because it is 
also used in writing a line to the modem), there is no RETRY_SEND any more
2. the user needs to rely upon the sms-resend-retry mechanism - this 
works fine AFAIK. E.g. put sms-resend-retry = 5 in the conf file. 
Without such a line, kannel will not try to resend the SMS

Best regards,
Andrija

P.S.

Long story short, the problem is in forcing the modem to send an SMS -
even if it reports an error.

In the official Kannel version, there is a RETRY_SEND (equal to 3, in
smsc_at.h) define that defines how many times an SMS sending process
needs to be retried. It's  purpose is illogical - having a complete and
logical sms-resend-retry mechanism built in the Kannel logic.

In the official Kannel version, if the software receives an error
response while trying to send the SMS, it simply tries again in a loop
(3 times, due to the value of RETRY_SEND). Such a behavior confuses the
GSM modem - on the first forcible SMS sending retry.

If the software should report unsuccessful SMS sending through
 bb_smscconn_send_failed(privdata->conn, msg,
SMSCCONN_FAILED_TEMPORARILY, octstr_create("ERROR"));

and let the modem "relax" (by asking it if there are new incoming SMS
messages pending) when it receives an error response from the modem, I
have seen that Siemens modems (TC 45, MC 39i) would never expose the
notorious issue.

I have never encountered the issue in the last 15 months - since I
altered the logic within smsc_at.c. With the RETRY_SEND mechanism, I
have had the issue more than once daily (I operate the GSM modems in a
rather intense SMS traffic).

Hope this comment will lead to a usable official version of smsc_at.c in
the future.


===========================================

Hillel wrote:
>
> Hi,
>
>  
>
> Andreas is keen to install it, but wants it to be re-sent.
>
>  
>
> Please put in the subject the word PATCH as otherwise it's not clear 
> which mail is the patch.
>
>  
>
> thanks
>
>  
>
> *From:* Andreas Fink [mailto:[email protected]]
> *Sent:* 19 September 2007 11:48 AM
> *To:* Hillel
> *Subject:* Re: Feature request/wish for Kannel
>
>  
>
> ok so you want me to go through houndreds of mails to find the patch?
>
>  
>
>  
>
> On 19.09.2007, at 10:24, Hillel wrote:
>
>  
>
> Hi Andreas,
>
>  
>
> I think you should read what Andrija Petrovic wrote as it seems there are
>
> problems with both AT+CNMI operation and AT+CMGR.
>
>  
>
> Both patches seem to have been submitted but never implemented.
>
>  
>
> Rgds
>
>  
>
> From: Andrija Petrovic [mailto:[email protected]] 
>
> Sent: 19 September 2007 10:09 AM
>
> To: [email protected] <mailto:[email protected]>
>
> Subject: Re: Feature request/wish for Kannel
>
>  
>
> Hi,
>
>  
>
> I'd better not answer directly to your direct question. Direct answer is
>
> "this patch has nothing to do with SIM card being filled up, but applying
>
> this patch will stop the SIM card from being filled up" :).
>
>  
>
> A complete answer should take into account the mode of operation of the AT
>
> modem.
>
>  
>
> 1. If the mode of operation is AT+CNMI (waiting for new message
>
> notifications), there is no reason why the modem should fill up the SIM
>
> card. Of course - with the current (unstable) version of smsc_at the SIM
>
> card used to fill up. Therefore, when I started working with Kannel, SIM
>
> overfilling was my primary concern. It turned out that once the modem
>
> started working correctly, with no "freezing" due to SMS send misbehavior,
>
> the SIM card never received a single SMS. All SMSes were consumed by the
>
> AT+CNMI mechanism (through +CMT notifications). When I say "never", that
>
> means 15 months on 3-4 modems synchronously, receiving up to 940 
> SMSes/day,
>
> with peaks up to 30 SMSes/minute. (the figure of 30 SMSes/minute may be
>
> another topic for discussion, due to the hard limit of 10 SMSes/minute for
>
> sending...)
>
>  
>
> 2. If the mode of operation is AT+CMGR (reading the SMSes that reached the
>
> SIM card), my actual patch did not address this problem. This part of
>
> smsc_at will be a matter of my next patch. Mr Andreas Fink did not take
>
> advantage of the AT+CMGL command (listing all the SMSes in the SIM card).
>
> That command speeds up the process of reading the SMSes from the SIM 
> card by
>
> at least an order of magnitude. In May of 2006 I submitted a rather large
>
> patch that solved this problem, but the patch never made it to submission.
>
> So, if you ask me, do not use the sim-buffering option for now.
>
>  
>
>  
>
> Best regards,
>
> Andrija Petrovic
>
>  
>
> P.S. Just as a curiosity - what do you use Kannel for? What is the
>
> exploitation scenario? Please, do not answer if it is a business secret or
>
> any type of confidential information.
>
>  
>
>  
>
> Hillel wrote: 
>
> Hi,
>
>  
>
> Will your patch fix the problem with the SIM card filling up and not being
>
> cleared out and so new SMSs not being able to come in?
>
>  
>
> Thanks
>
>  
>
>  
>
>  
>
>  
>
>  
>
> Andreas Fink
>
>  
>
> Fink Consulting GmbH
>
> Global Networks Schweiz AG
>
> BebbiCell AG
>
>  
>
> ---------------------------------------------------------------
>
> Tel: +41-61-6666330 Fax: +41-61-6666331  Mobile: +41-79-2457333
>
> Address: Clarastrasse 3, 4058 Basel, Switzerland
>
> E-Mail:  [email protected] <mailto:[email protected]>
>
> www.finkconsulting.com www.global-networks.ch www.bebbicell 
> <http://www.bebbicell>.ch
>
> ---------------------------------------------------------------
>
> ICQ: 8239353 MSN: [email protected] <mailto:[email protected]> AIM: smsrelay 
> Skype: andreasfink
>
> Yahoo: finkconsulting SMS: +41792457333
>
>  
>
>  
>
>  
>
>  
>
>  
>
smsc_at.diff (text/plain, 10.2 KB)
Index: gw/smsc/smsc_at.c
===================================================================
RCS file: /home/cvs/gateway/gw/smsc/smsc_at.c,v
retrieving revision 1.37
diff -u -r1.37 smsc_at.c
--- gw/smsc/smsc_at.c	22 Jul 2007 13:15:25 -0000	1.37
+++ gw/smsc/smsc_at.c	11 Sep 2007 10:34:18 -0000
@@ -339,7 +339,7 @@
         errno = 0;
         s = write(privdata->fd, octstr_get_cstr(linestr) + data_written, 
                   count - data_written);
-        if (s < 0 && errno == EAGAIN && write_count < RETRY_SEND) {
+        if (s < 0 && errno == EAGAIN && write_count < RETRY_WRITE) {
             gwthread_sleep(1);
             ++write_count;
         } else if (s > 0) {
@@ -372,7 +372,7 @@
     while (1) {
         errno = 0;
         s = write(privdata->fd, ctrlz, 1);
-        if (s < 0 && errno == EAGAIN && write_count < RETRY_SEND) {
+        if (s < 0 && errno == EAGAIN && write_count < RETRY_WRITE) {
             gwthread_sleep(1);
             ++write_count;
         } else
@@ -401,7 +401,7 @@
     count = strlen(line);
     while(count > data_written) {
         s = write(privdata->fd, line + data_written, count - data_written);
-        if (s < 0 && errno == EAGAIN && write_count < RETRY_SEND) {
+        if (s < 0 && errno == EAGAIN && write_count < RETRY_WRITE) {
             gwthread_sleep(1);
             ++write_count;
         } else if (s > 0) {
@@ -1188,12 +1188,7 @@
 
     idle_timeout = 0;
     while (!privdata->shutdown) {
-        l = gw_prioqueue_len(privdata->outgoing_queue);
-        if (l > 0) {
-            at2_send_messages(privdata);
-            idle_timeout = time(NULL);
-        } else
-            at2_wait_modem_command(privdata, 1, 0, NULL);
+				at2_wait_modem_command(privdata, 1, 0, NULL);
 
         /* read error, so re-connect */
         if (privdata->fd == -1) {
@@ -1225,6 +1220,11 @@
             }
             memory_poll_timeout = time(NULL);
         }
+
+        if (gw_prioqueue_len(privdata->outgoing_queue) > 0) {
+            at2_send_messages(privdata);
+            idle_timeout = time(NULL);
+        }
     }
     at2_close_device(privdata);
     mutex_lock(conn->flow_mutex);
@@ -1991,14 +1991,12 @@
 {
     Msg *msg;
 
-    do {
-        if (privdata->modem->enable_mms && 
-            gw_prioqueue_len(privdata->outgoing_queue) > 1)
-            at2_send_modem_command(privdata, "AT+CMMS=2", 0, 0);
-
-        if ((msg = gw_prioqueue_remove(privdata->outgoing_queue)))
-            at2_send_one_message(privdata, msg);
-    } while (msg);
+		if (privdata->modem->enable_mms && 
+				gw_prioqueue_len(privdata->outgoing_queue) > 1)
+				at2_send_modem_command(privdata, "AT+CMMS=2", 0, 0);
+
+		if ((msg = gw_prioqueue_remove(privdata->outgoing_queue)))
+				at2_send_one_message(privdata, msg);
 }
 
 
@@ -2007,7 +2005,6 @@
     unsigned char command[500];
     int ret = -1;
     char sc[3];
-    int retries = RETRY_SEND;
 
     if (octstr_len(privdata->my_number)) {
         octstr_destroy(msg->sms.sender);
@@ -2033,103 +2030,90 @@
             return;
         }	
 
-        ret = -99;
-        retries = RETRY_SEND;
-        while ((ret != 0) && (retries-- > 0)) {
-            int msg_id = -1;
-            /* 
-             * send the initial command and then wait for > 
-             */
-            sprintf(command, "AT+CMGS=%ld", octstr_len(pdu) / 2);
-            
-            ret = at2_send_modem_command(privdata, command, 5, 1);
-            debug("bb.smsc.at2", 0, "AT2[%s]: send command status: %d",
-                  octstr_get_cstr(privdata->name), ret);
-
-            if (ret != 1) /* > only! */
-                continue;
-                
-            /* 
-             * Ok the > has been see now so we can send the PDU now and a 
-             * control Z but no CR or LF 
-             * 
-             * We will handle the 'nokiaphone' types a bit differently, since
-             * they have a generic error in accepting PDUs that are "too big".
-             * Which means, PDU that are longer then 18 bytes get truncated by
-             * the phone modems. We'll buffer the PDU output in a loop.
-             * All other types will get handled as used to be.
-             */
-
-            if (octstr_compare(privdata->modem->id, octstr_imm("nokiaphone")) != 0) {           
-            
-                sprintf(command, "%s%s", sc, octstr_get_cstr(pdu));
-                at2_write(privdata, command);
-                at2_write_ctrlz(privdata);
-            
-            } else {
-            
-                /* include the CTRL-Z in the PDU string */
-                sprintf(command, "%s%s%c", sc, octstr_get_cstr(pdu), 0x1A);
-       
-                /* chop PDU into 18-byte-at-a-time pieces to prevent choking 
-                 * of certain GSM Phones (e.g. Nokia 6310, 6230 etc.) */
-                if (strlen(command) > 18) {
-                    unsigned char chop[20];
-                    int len = strlen(command);
-                    int pos = 0;
-                    int ret = 18;
-
-                    while (pos < len) {
-                        if (pos + ret > len)
-                            ret = len - pos;
-                        memcpy(chop, command + pos, ret);
-                        pos += ret;
-                        chop[ret] = '\0';
-                        at2_write(privdata, chop);
-                        gwthread_sleep((double) 10/1000);
-                    }
-                } else {
-                    at2_write(privdata, command);
-                }
-            }               
-                
-            /* wait 20 secs for modem command */
-            ret = at2_wait_modem_command(privdata, 20, 0, &msg_id);
-            debug("bb.smsc.at2", 0, "AT2[%s]: send command status: %d",
-                  octstr_get_cstr(privdata->name), ret);
-
-            if (ret != 0) /* OK only */
-                continue;
-
-            /* store DLR message if needed for SMSC generated delivery reports */
-            if (DLR_IS_ENABLED_DEVICE(msg->sms.dlr_mask)) {
-                if (msg_id == -1)
-                    error(0,"AT2[%s]: delivery notification requested, but I have no message ID!",
-                          octstr_get_cstr(privdata->name));
-                else {
-                    Octstr *dlrmsgid = octstr_format("%d", msg_id);
-
-                    dlr_add(privdata->conn->id, dlrmsgid, msg);
-
-                    O_DESTROY(dlrmsgid);
-                }
-            }
-
-            bb_smscconn_sent(privdata->conn, msg, NULL);
-        }
-
-        if (ret != 0) {
-            /*
-             * no need to do counter_increase(privdata->conn->failed) here,
-             * since bb_smscconn_send_failed() will inc the counter on
-             * SMSCCONN_FAILED_MALFORMED
-             */
-            bb_smscconn_send_failed(privdata->conn, msg,
-	        SMSCCONN_FAILED_MALFORMED, octstr_create("MALFORMED"));
-        }
-
-        O_DESTROY(pdu);
-    }
+				int msg_id = -1;
+				/* 
+				 * send the initial command and then wait for > 
+				 */
+				sprintf(command, "AT+CMGS=%ld", octstr_len(pdu) / 2);
+				
+				ret = at2_send_modem_command(privdata, command, 5, 1);
+				debug("bb.smsc.at2", 0, "AT2[%s]: send command status: %d",
+							octstr_get_cstr(privdata->name), ret);
+
+				if (ret == 1) {/* > only! */
+						
+					/* 
+					 * Ok the > has been see now so we can send the PDU now and a 
+					 * control Z but no CR or LF 
+					 * 
+					 * We will handle the 'nokiaphone' types a bit differently, since
+					 * they have a generic error in accepting PDUs that are "too big".
+					 * Which means, PDU that are longer then 18 bytes get truncated by
+					 * the phone modems. We'll buffer the PDU output in a loop.
+					 * All other types will get handled as used to be.
+					 */
+
+					if (octstr_compare(privdata->modem->id, octstr_imm("nokiaphone")) != 0) {           
+					
+							sprintf(command, "%s%s", sc, octstr_get_cstr(pdu));
+							at2_write(privdata, command);
+							at2_write_ctrlz(privdata);
+					
+					} else {
+					
+							/* include the CTRL-Z in the PDU string */
+							sprintf(command, "%s%s%c", sc, octstr_get_cstr(pdu), 0x1A);
+		 
+							/* chop PDU into 18-byte-at-a-time pieces to prevent choking 
+							 * of certain GSM Phones (e.g. Nokia 6310, 6230 etc.) */
+							if (strlen(command) > 18) {
+									unsigned char chop[20];
+									int len = strlen(command);
+									int pos = 0;
+									int ret = 18;
+
+									while (pos < len) {
+											if (pos + ret > len)
+													ret = len - pos;
+											memcpy(chop, command + pos, ret);
+											pos += ret;
+											chop[ret] = '\0';
+											at2_write(privdata, chop);
+											gwthread_sleep((double) 10/1000);
+									}
+							} else {
+									at2_write(privdata, command);
+							}
+					}               
+							
+					/* wait 20 secs for modem command */
+					ret = at2_wait_modem_command(privdata, 20, 0, &msg_id);
+					debug("bb.smsc.at2", 0, "AT2[%s]: send command status: %d",
+								octstr_get_cstr(privdata->name), ret);
+
+					if (ret != 0) {
+							bb_smscconn_send_failed(privdata->conn, msg,
+						SMSCCONN_FAILED_TEMPORARILY, octstr_create("ERROR"));
+					}else{
+						/* store DLR message if needed for SMSC generated delivery reports */
+						if (DLR_IS_ENABLED_DEVICE(msg->sms.dlr_mask)) {
+								if (msg_id == -1)
+										error(0,"AT2[%s]: delivery notification requested, but I have no message ID!",
+													octstr_get_cstr(privdata->name));
+								else {
+										Octstr *dlrmsgid = octstr_format("%d", msg_id);
+
+										dlr_add(privdata->conn->id, dlrmsgid, msg);
+
+										O_DESTROY(dlrmsgid);
+								}
+						}
+
+						bb_smscconn_sent(privdata->conn, msg, NULL);
+					}
+				}
+				O_DESTROY(pdu);
+		}
 }
 
 
Index: gw/smsc/smsc_at.h
===================================================================
RCS file: /home/cvs/gateway/gw/smsc/smsc_at.h,v
retrieving revision 1.12
diff -u -r1.12 smsc_at.h
--- gw/smsc/smsc_at.h	7 Jan 2007 23:52:53 -0000	1.12
+++ gw/smsc/smsc_at.h	11 Sep 2007 10:34:18 -0000
@@ -82,8 +82,8 @@
 #define PNT_INTER       1
 #define PNT_NATIONAL    2
 
-/* The number of times to attempt to send a message should sending fail */
-#define RETRY_SEND 3
+/* The number of times to attempt to write a line should writing fail */
+#define RETRY_WRITE 3
 
 /* 
  * defines for use with the so-called "SIM buffering techinique":
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.