Re: dlr_mysql_add and internal charset

Peter Christensen wrote:

> Hi,
> 
> I'd like to address a couple things now that a new kannel release near:
> 
> 
> 1. Some while ago, I reported problems within the dlr_mysql_add 
> function. If the entry->timestamp, entry->source, or entry->url contains 
> some unfortunate characters (most significantly <'>), the SQL query gets 
> broken and the DLRs are wasted.
> After a while, the first patch was submitted, but as it used 
> mysql_real_escape_string, it would potentially require an additional 
> MySQL connection (or something - don't remember what the exact problem 
> was), so it was not committed, and another patch was promised in a near 
> future. Apparently this patch never came, however, and I see that the 
> current CVS is still not escaping the strings.

correct, I have that escaping version of mysql here and it's scheduled for 
commit to 1.5.0 devel, since I won't have the time to test it that extensively 
to ensure stability for 1.4.1 stable.

Any thoughts on this?

I could post the patch and let the list confirm via votes that it should or 
should not go to 1.4.1 stable?

> So, question is: What is the cause of the delay, and are you interested 
> in my own patch, which uses mysql_escape_string instead of 
> mysql_real_escape_string?

The cuase is lack in tme... at least from my side... I'm pushing, but I have 
several "construction sides" at Kannel side, including the more "important" 
WTP-SAR issue.

If you have ready code for that post to the mailing list. I will do the same, so 
  we can leaverage work time.

> I admit that problems related to this is somewhat rare (most likely 
> cause for errors would be an SMS with <'> in the originator string), but 
> since it can be fixed relatively easy, I see no reason why NOT to do it. 
> Although mysql_escape_string does not look at the character set (unlike 
> mysql_real_escape_string), I still believe that it will be better to use 
> it than to do nothing.

absolutely agree'ing... following the main strategy: stability and safe.

> 2. I've noticed that the charset_gsm_to_latin1 and charset_latin1_to_gsm 
> functions actually uses windows-1252 as character set, but at least the 
> SMPP gateway uses iso-8859-1 internally, which practically removes 
> support for the € (euro) sign. This is probably also a problem within 
> other gateways. If interesseted, I can supply a patch.

yep... Please comment the patch in detail, so that anyone of us who is reviewing 
has an easier job.

> But in relation to that, there is one thing which have begun to annoy 
> me. By restricting to the windows-1252 character set when transmitting 
> with the GSM character set, you remove support for the 10 Greek 
> characters which is support by GSM. And as it always is when something 
> is potentially possible, there will always be some large annoying 
> customer who wants support for that particular feature.
> This last bit is only an observation and nothing more. I realize that 
> fixing this (preferable by using UTF-8 or UCS-2BE as internal character 
> set regardless of the output character set), would require a 
> significantly amount of recoding within almost every part of the kannel 
> software package.

yep.

> The reason while I have not attached any patches to this mail (although 
> I have made quite a few changes), is that my own patch is really a 
> combination of several patches, which is not in sync with the CVS. But 
> if people are interested, I will update the patch and post the relevant 
> bits.

yes, please try to keep clean logical patchsets. Which means one patchset = one 
logical change. It's too hard to trace several issues in just one single patchset.

> The whole patch does the following:
>  SMPP:
>   * Add support for mBlox operator and billing identifier (Not my own work)
>   * Add support for ALL SMPPv5 error codes, including mBlox specific 
> codes. (That is, recognize them and translate to human readable text)
>   * Use CP1252 instead of ISO-8859-1 as internal charset
>   * Added pack_udh parameter. When set, messages are sent as packed GSM 
> data when UDH is present. (A few gateways require this)
> 
>  MySQL:
>   * Escapes strings in dlr_mysql_add
> 
>  run_kannel_box:
>   * Added waitpid after kill to avoid false terminations (the init 
> scripts reports that kannel is terminated, while bearerbox is actually 
> stuck within a connect call. - happens relatively often). With later CVS 
> releases this didn't seem to work, so instead I've added some 
> functionaly to the init script instead.

Stipe


-- 
Für weitere Fragen stehe ich Ihnen gerne telefonisch
oder via E-Mail zur Verfügung.

Mit freundlichen Grüßen


Stipe Tolj
OSS System Architect

-------------------------------------------------------------------
Kölner Landstrasse 419
40589 Düsseldorf, NRW, Germany

tolj.org system architecture      Kannel Software Foundation (KSF)
http://www.tolj.org/              http://www.kannel.org/

mailto:st_{at}_tolj.org           mailto:stolj_{at}_kannel.org
-------------------------------------------------------------------