[RFC] new module : SOAP.
"Oded Arbel" <[email protected]>
| Newsgroups | gmane.comp.mobile.kannel.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi list.
Created a new SMSC module for Kannel for use with connection
to some proprietary gateway systems some of our providers use
(we currently use this module for 3 different providers). I
call this SOAP, after the Microsoft "standard" as it seems to
me to be the same thing though none of our providers call it
like this in their specs. In actuality, simple XML over HTTP,
storing a provider specific XML format in a POST variable.
I tried to make this module as general as possible, but as
each provider uses different semantics and different XML
formats and even different data formats inside the XML for
the same message elements, it's harder then it seems at
first. The approach I've taken so far is to use some
configuration variables (added to cfg.def - patch attached),
file templates for creating the text buffers Kannel sends,
"spec" files for parsing the XML buffers Kannel receives, and
some hard coded "keywords" inside the template parser to
generate some weird data representations that some of our
providers like to use.
The module's HTTP server (for calls from the SOAP server to
Kannel for MO and DLR) exposes two "calls" :
http://<ip>:<port>/mo for MO delivery and
http://<ip>:<port>/dlr for DLR delivery. the server expects a
singe CGI post parameter whose name doesn't matter (it is
discarded) and then reads and attempts to parse the XML using
the relevant "spec" file. the syntax to the spec file is very
simple: it's a tab separated list where each record (line)
contains 2 or 3 elements -
1) identification of element we wish to extract. sorry to
point you to the code, but these keywords must conform to
hard coded values found in the code itself. I tried to
conform to the SMS message structure defined in Kannel's
msg-decl where possible
2) path in the XML hierarchy to the element
3) if the required data is not the content of the XML element
but one of its attributes, then this will contain the
required attribute's name
The XML template parser understand various "tokens" that can
be embeded in the templates to provide the actuall run-time
content. you can either use the format %token_name or %{token
name} . if you are not using the curly braced version then
the token name must start with a letter and continue only
with letters. numers and under-scores. if the token is
encapsulated is culry braces it can contain anything (except
and ending curly brace), but it of course wouldn't make much
sense unless it conforms to the same format as the former
version as they both match against the same keywords. any
member name of the SMS message structure can be used, and
also some special "data formats" are recognized - look in the
soap_parse_token() function's code.
Sorry for no documentation yet, I will get down to it (of
course - I will be much more motivated to provide
documentation if people will show interest in the module ;-),
in the mean time you can always read the source.
I had to add several new functions to the Kannel
infrastructures (patches attached), and here they are described :
* date.[ch]:
int date_parse_iso (struct universaltime *ut, Octstr *os) -
attempts to parse an ISO-8601 date representation,
substituting missing fields with the base values (0 or 1),
returning 0 on success and -1 on failure. the time is read
from the Octstr os and returned in the universal time structure ut.
Octstr* date_create_iso(time_t unixtime) - creates an
ISO-8601 formatted date string from a unix epoch time stamp.
* charset.[ch]:
int charset_convert(Octstr* string, char* charset_from, char*
charset_to) - converts a string buffer between arbitary
character sets using the standard glibc library iconv
(Hopefully it is present in other OSs). I had to add this
utility as the libxml encoding conversions do not cover
nearly as many character sets as I need to support (strangely
enough - some providers use XML and not utf-8).
* configure.in config.h.in
checking for iconv to support charset_convert() in charset.[ch]
* msg-decl.h, msg.[ch]
Changed the msg id used in sms and ack types to 64 bit
integer. I know this will probably be very controversial and
is the number one compatibility breaker here, but _all_ of
the providers we're dealing with here use 64bit integers for
their message IDs and I have to correlate to that, otherwise
I can lose synchronization between messages and acks. it
probably has something to do with java and it's 64 bit native
integers - apparently all of our providers use Java as their
programming language for the XML interfaces.
The most important lesson - after applying this patch your
store files will no longer work and cause bearerbox to crash
- so make sure to delete the store files before running a
modified bearerbox. this change would probably turn-off most
of the people that would have otherwise tested this module,
but I do hope that you will take a look at it anyway. I do
think that changing the message IDs to 64 bit is something we
should have done much earlier.
* utils.[ch]
unsigned long long decode_network_int64(unsigned char *data)
, void encode_network_int64(unsigned char *data, unsigned
long long value) - support for 64 bit integers
unsigned long long gw_generate_id() - can be used to generate
pseudo-unique 64 bit integers for id purposes
notes:
- HTTPS calls are handled using the normal Kannel HTTP
infrastructure, so HTTPS calls that require client
certificates and the HTTPS server using the new
receive-port-ssl config parameter use the core configuration
definitions ssl-client-certkey-file, ssl-server-cert-file and
ssl-server-key-file.
- in the XML examples you'll notice a "billing" paremeter, as
this conforms with the billing patch I submitted to the list
a while ago. if the "tariff" patch is accepted then the fix
would be simply to change the text in the file from "billing"
to "tariff"
Cheers
--
Oded Arbel
m-Wise mobile solutions
[email protected]
+972-9-9581711 (116)
+972-67-340014
::..
Time is an illusion. Lunchtime doubly so.
--Ford Prefect
soap.patch
(application/octet-stream, 12 KB)
--- gwlib/date.c 2002-04-07 16:43:43.000000000 +0300
+++ gwlib/date.c 2002-08-02 13:55:17.000000000 +0300
@@ -168,6 +168,58 @@
return -1;
}
+int date_parse_iso (struct universaltime *ut, Octstr *os)
+{
+ long pos = 0;
+ int c;
+
+ /* assign defaults */
+ ut->month = 0;
+ ut->day = 1;
+ ut->hour = 0;
+ ut->minute = 0;
+ ut->second = 0;
+
+ if ((pos = octstr_parse_long(&(ut->year), os, pos, 10)) < 0)
+ return -1;
+ if (ut->year < 70)
+ ut->year += 2000;
+ else if (ut->year < 100)
+ ut->year += 1900;
+
+ while ((c = octstr_get_char(os, pos++)) != -1 && !gw_isdigit(c));
+ if ((pos = octstr_parse_long(&(ut->month), os, pos, 10)) < 0)
+ return 0;
+
+ while ((c = octstr_get_char(os, pos++)) != -1 && !gw_isdigit(c));
+ if ((pos = octstr_parse_long(&(ut->day), os, pos, 10)) < 0)
+ return 0;
+
+ while ((c = octstr_get_char(os, pos++)) != -1 && !gw_isdigit(c));
+ if ((pos = octstr_parse_long(&(ut->hour), os, pos, 10)) < 0)
+ return 0;
+
+ while ((c = octstr_get_char(os, pos++)) != -1 && !gw_isdigit(c));
+ if ((pos = octstr_parse_long(&(ut->minute), os, pos, 10)) < 0)
+ return 0;
+
+ while ((c = octstr_get_char(os, pos++)) != -1 && !gw_isdigit(c));
+ if ((pos = octstr_parse_long(&(ut->second), os, pos, 10)) < 0)
+ return 0;
+
+ return 0;
+}
+
+Octstr* date_create_iso(time_t unixtime)
+{
+ struct tm tm;
+
+ tm = gw_gmtime((time_t) unixtime);
+
+ return octstr_format("%d-%02d-%02d %02d:%02d:%02d",
+ tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec);
+}
+
/* Note that this implementation makes unportable assumptions about time_t. */
long date_universal_now(void)
--- kannel-dev/gateway/gwlib/date.h 2001-05-03 17:38:41.000000000 +0300
+++ gateway/gwlib/date.h 2002-06-30 12:44:47.000000000 +0300
@@ -45,8 +45,23 @@
*/
long date_parse_http(Octstr *date);
+/*
+ * attempt to read an ISO-8601 format or similar, making no assumptions on
+ * seperators and number of elements, adding 0 or 1 to missing fields
+ * For example, acceptable formats :
+ * 2002-05-15 13:23:44
+ * 02/05/15:13:23
+ * support of 2 digit years is done by assuming years 70 an over are 20th century. this will
+ * have to be revised sometime in the next 50 or so years
+ */
+int date_parse_iso(struct universaltime *ut, Octstr *os);
/*
+ * create an ISO-8601 formated time stamp
+ */
+Octstr* date_create_iso(time_t unixtime);
+
+/*
* Return the current date and time as a unix time value.
*/
long date_universal_now(void);
--- gwlib/cfg.def 2002-08-07 10:56:45.000000000 +0300
+++ gwlib/cfg.def 2002-08-07 11:01:17.000000000 +0300
@@ -179,6 +179,7 @@
OCTSTR(our-host)
OCTSTR(our-port)
OCTSTR(receive-port)
+ OCTSTR(receive-port-ssl)
OCTSTR(connect-allow-ip)
OCTSTR(system-id)
OCTSTR(system-type)
@@ -219,6 +220,10 @@
OCTSTR(priority)
OCTSTR(notification-pid)
OCTSTR(notification-addr)
+ OCTSTR(form-variable)
+ OCTSTR(form-urlencoded)
+ OCTSTR(xml-files)
+ OCTSTR(xmlspec-files)
)
--- gwlib/charset.c 2002-01-25 13:49:05.000000000 +0200
+++ gwlib/charset.c 2002-08-02 13:55:17.000000000 +0300
@@ -7,6 +7,11 @@
*/
#include "gwlib/gwlib.h"
+#include <errno.h>
+
+#if HAVE_ICONV_H
+#include <iconv.h>
+#endif
/* Map GSM default alphabet characters to ISO-Latin-1 characters.
* The greek characters at positions 16 and 18 through 26 are not
@@ -401,3 +406,48 @@
return ret;
}
+
+int charset_convert(Octstr* string, char* charset_from, char* charset_to)
+{
+#if HAVE_ICONV_H
+ char *from_buf, *to_buf, *pointer;
+ size_t inbytes, outbytes, ret;
+ iconv_t cd;
+
+ if (!charset_from || !charset_to || !string) /* sanity check */
+ return -1;
+
+ cd = iconv_open(charset_to, charset_from);
+ /* Did I succeed in getting a conversion descriptor ? */
+ if (cd == (iconv_t)(-1)) {
+ /* I guess not */
+ error(0,"Failed to convert string from %s to %s - probably broken type names.",
+ charset_from, charset_to);
+ return -1;
+ }
+ from_buf = octstr_get_cstr(string);
+ /* allocate max sized buffer, assuming target encoding may be 4 byte unicode */
+ inbytes = octstr_len(string);
+ outbytes = sizeof(char) * octstr_len(string) * 4;
+ pointer = to_buf = gw_malloc(outbytes + 1);
+ memset(to_buf,0,outbytes+1);
+ ret = iconv(cd, (const char**)&from_buf, &inbytes, &pointer, &outbytes);
+ iconv_close(cd);
+ if (ret != -1) {
+ /* conversion succeeded */
+ octstr_delete(string,0,octstr_len(string));
+ octstr_append_cstr(string, to_buf);
+ if (ret)
+ debug("charset",0,"charset_convert did %d non-reversible conversions",ret);
+ ret = 0;
+ } else
+ error(0,"Failed to convert string from %s to %s, errno: %d",charset_from, charset_to, errno);
+ if (errno == EILSEQ)
+ {
+ debug("charset_convert",0,"found an invalid multibyte sequence at position %d",from_buf - octstr_get_cstr(string));
+ }
+ gw_free(to_buf);
+ return ret;
+#endif
+ return 0;
+}
--- gwlib/charset.h 2001-12-07 15:48:57.000000000 +0200
+++ gwlib/charset.h 2002-06-25 17:18:07.000000000 +0300
@@ -72,4 +72,9 @@
*/
int charset_from_utf8(Octstr *utf8, Octstr **to, Octstr *charset_to);
+/* use iconv library to convert an Octstr in place, from source character set to
+ * destination character set
+ */
+int charset_convert(Octstr* string, char* charset_from, char* charset_to);
+
#endif
--- configure.in 2002-08-07 10:56:44.000000000 +0300
+++ configure.in 2002-06-16 16:14:36.000000000 +0300
@@ -95,7 +81,7 @@
AC_HEADER_STDC
AC_CHECK_HEADERS(sys/ioctl.h sys/time.h sys/types.h unistd.h sys/poll.h)
-AC_CHECK_HEADERS(pthread.h getopt.h syslog.h)
+AC_CHECK_HEADERS(pthread.h getopt.h syslog.h iconv.h)
dnl Checks for typedefs, structures, and compiler characteristics.
--- config.h.in 2002-08-07 10:56:44.000000000 +0300
+++ config.h.in 2002-06-16 16:18:00.000000000 +0300
@@ -113,6 +113,9 @@
/* Define if you have <syslog.h>. */
#undef HAVE_SYSLOG_H
+/* Define if you have <iconv.h>. */
+#undef HAVE_ICONV_H
+
/* Define if you have and want to use the ssl library (-lssl) */
#undef HAVE_LIBSSL
--- gw/msg-decl.h 2002-08-07 10:56:44.000000000 +0300
+++ gw/msg-decl.h 2002-07-19 16:48:15.000000000 +0300
@@ -30,7 +30,7 @@
OCTSTR(smsc_id);
OCTSTR(service);
OCTSTR(account);
- INTEGER(id);
+ INT64(id);
INTEGER(sms_type);
INTEGER(mclass);
INTEGER(mwi);
@@ -50,7 +50,7 @@
{
INTEGER(nack);
INTEGER(time);
- INTEGER(id);
+ INT64(id);
})
MSG(wdp_datagram,
@@ -64,4 +64,5 @@
#undef MSG
#undef INTEGER
+#undef INT64
#undef OCTSTR
--- gw/msg.c 2001-07-16 16:41:05.000000000 +0300
+++ gw/msg.c 2002-06-30 11:56:05.000000000 +0300
@@ -20,9 +20,11 @@
*/
static void append_integer(Octstr *os, long i);
+static void append_int64(Octstr *os, unsigned long long i);
static void append_string(Octstr *os, Octstr *field);
static int parse_integer(long *i, Octstr *packed, int *off);
+static int parse_int64(long long *i, Octstr *packed, int *off);
static int parse_string(Octstr **os, Octstr *packed, int *off);
static char *type_as_str(Msg *msg);
@@ -40,6 +42,7 @@
msg->type = type;
#define INTEGER(name) p->name = 0
+#define INT64(name) p->name = 0
#define OCTSTR(name) p->name = NULL
#define MSG(type, stmt) { struct type *p = &msg->type; stmt }
#include "msg-decl.h"
@@ -54,6 +57,7 @@
new = msg_create(msg->type);
#define INTEGER(name) p->name = q->name
+#define INT64(name) p->name = q->name
#define OCTSTR(name) \
if (q->name == NULL) p->name = NULL; \
else p->name = octstr_duplicate(q->name);
@@ -72,6 +76,7 @@
return;
#define INTEGER(name) p->name = 0
+#define INT64(name) p->name = 0
#define OCTSTR(name) octstr_destroy(p->name)
#define MSG(type, stmt) { struct type *p = &msg->type; stmt }
#include "msg-decl.h"
@@ -90,6 +95,8 @@
debug("gw.msg", 0, "%*s type: %s", level, "", type_as_str(msg));
#define INTEGER(name) \
debug("gw.msg", 0, "%*s %s.%s: %ld", level, "", t, #name, (long) p->name)
+#define INT64(name) \
+ debug("gw.msg", 0, "%*s %s.%s: %lld", level, "", t, #name, (long long) p->name)
#define OCTSTR(name) \
debug("gw.msg", 0, "%*s %s.%s:", level, "", t, #name); \
octstr_dump(p->name, level + 1)
@@ -114,6 +121,7 @@
append_integer(os, msg->type);
#define INTEGER(name) append_integer(os, p->name)
+#define INT64(name) append_int64(os, p->name)
#define OCTSTR(name) append_string(os, p->name)
#define MSG(type, stmt) \
case type: { struct type *p = &msg->type; stmt } break;
@@ -147,6 +155,8 @@
#define INTEGER(name) \
if (parse_integer(&(p->name), os, &off) == -1) goto error
+#define INT64(name) \
+ if (parse_int64(&(p->name), os, &off) == -1) goto error
#define OCTSTR(name) \
if (parse_string(&(p->name), os, &off) == -1) goto error
#define MSG(type, stmt) \
@@ -180,6 +190,14 @@
octstr_append_data(os, buf, 4);
}
+static void append_int64(Octstr *os, unsigned long long i)
+{
+ unsigned char buf[8];
+
+ encode_network_int64(buf, i);
+ octstr_append_data(os, buf, 8);
+}
+
static void append_string(Octstr *os, Octstr *field)
{
if (field == NULL)
@@ -207,6 +225,21 @@
return 0;
}
+static int parse_int64(long long *i, Octstr *packed, int *off)
+{
+ unsigned char buf[8];
+
+ gw_assert(*off >= 0);
+ if (*off + 8 > octstr_len(packed)) {
+ error(0, "Packet too short while unpacking Msg.");
+ return -1;
+ }
+
+ octstr_get_many_chars(buf, packed, *off, 8);
+ *i = decode_network_int64(buf);
+ *off += 8;
+ return 0;
+}
static int parse_string(Octstr **os, Octstr *packed, int *off)
{
--- gw/msg.h 2001-10-11 19:18:33.000000000 +0200
+++ gw/msg.h 2002-06-30 11:56:44.000000000 +0300
@@ -22,6 +22,7 @@
enum msg_type type;
#define INTEGER(name) long name
+ #define INT64(name) long long name
#define OCTSTR(name) Octstr *name
#define MSG(type, stmt) struct type stmt type;
#include "msg-decl.h"
--- gwlib/utils.c 2001-10-31 12:52:46.000000000 +0200
+++ gwlib/utils.c 2002-06-30 13:32:16.000000000 +0300
@@ -300,6 +300,26 @@
data[3] = value & 0xff;
}
+unsigned long long decode_network_int64(unsigned char *data) {
+ return
+ ((long long)data[0] << 56) | ((long long)data[1] << 48) |
+ ((long long)data[2] << 40) | ((long long)data[3] << 32) |
+ ((long long)data[4] << 24) | ((long long)data[5] << 16) |
+ ((long long)data[6] << 8) | (long long)data[7];
+}
+
+
+void encode_network_int64(unsigned char *data, unsigned long long value) {
+ data[0] = (value >> 56) & 0xff;
+ data[1] = (value >> 48) & 0xff;
+ data[2] = (value >> 40) & 0xff;
+ data[3] = (value >> 32) & 0xff;
+ data[4] = (value >> 24) & 0xff;
+ data[5] = (value >> 16) & 0xff;
+ data[6] = (value >> 8) & 0xff;
+ data[7] = value & 0xff;
+}
+
/* Something that does the same as GNU cfmakeraw. We don't use cfmakeraw
so that we always know what it does, and also to reduce configure.in
complexity. */
@@ -366,3 +386,14 @@
}
+unsigned long long gw_generate_id()
+{
+ /* create a 64 bit unique Id by putting a 32 bit epoch time value
+ * and a 32 bit random value together */
+ unsigned long random, timer;
+
+ random = gw_rand();
+ timer = (unsigned long)time(NULL);
+
+ return ((unsigned long long)timer << 32) + random;
+}
--- gwlib/utils.h 2001-03-19 10:51:28.000000000 +0200
+++ gwlib/utils.h 2002-06-30 13:03:54.000000000 +0300
@@ -95,6 +95,12 @@
*/
void encode_network_long(unsigned char *data, unsigned long value);
+/*
+ * Same as the previous two, but encodes and decodes 64 bit unsigned intergers
+ */
+unsigned long long decode_network_int64(unsigned char *data);
+void encode_network_int64(unsigned char *data, unsigned long long value);
+
/* kannel implementation of cfmakeraw, which is an extension in GNU libc */
void kannel_cfmakeraw (struct termios *tio);
@@ -114,5 +120,10 @@
*/
int roundup_div(int a, int b);
+/*
+ * generate a unique id
+ * (not guarenteed to be unique, but it's extremly unlikely for it not to be)
+ */
+unsigned long long gw_generate_id();
#endif
providerxmlfiles.zip
(application/x-zip-compressed, 1.7 KB) - not displayed
kannel.conf.provider
(application/octet-stream, 441 B) - not displayed
smsc_soap.c.gz
(application/x-gzip, 16.5 KB) - not displayed