Re: smsbox filtering on GSM 03.38 even when alt_charset is defined

 Hi Alexander

I think we have a disconnect here. Let me try to explain what I am saying.

>  encoding=0 is UTF-8 in kannel and not LATIN1 anymore

Not true. It can be ISO-8859-1 or UTF-8. See this link (we have
confirmed it to be true).

http://webcache.googleusercontent.com/search?q=cache:NzDV7eArI9IJ:www.linkas.it/docs/kannel/Characters,%2520Kannel%2520%26%2520MT%2520SMS.docx+kannel+hebrew&cd=7&hl=en&ct=clnk&gl=us&client=firefox-a

> if you send LATIN1 and expect all chars to arrive on GSM charset then
you are wrong

I may not be. We are not sending to the handset directly. The charset
you send over SMPP depends on what the SMSC wants. From the client side
(to smsbox), I am sending coding=0, and sending data in ISO-8859-1
charset. At the same time I have alt_charset=ISO-8859-1. What I expect
is, going out to the SMSC, data_coding=0 and data in charset ISO-8859-1.

> Handset simple unable to display chars that are not in GSM03.38

True enough. But we are not sending directly to the handset. This
particular SMSC requires us to send ISO-8859-1 charset for
data_coding=0. They do their own transformations for each operator
(which may or may not be GSM 03.38). And that is what the alt_charset
setting in Kannel is for. To accommodate such non-standard SMSCs.

>  if you read this function then you will see

Yes I have seen the code. In fact, that was what I was referring to in
my initial post.

>  If you wish to send chars that are not in GSM03.38 you have to use
UCS2 coding

Well, that shouldn't be. I want to be able to send to SMSC ISO-8859-1
with data_coding=0. The parameter alt_charset was (seems like) put in
place to allow this. But the piece of code we are talking about here, is
preventing that. Hence, my insistence that it is a bug.

Sanjay

On 9/22/2010 3:42 AM, Alexander Malysh wrote:
> Hi,
>
> encoding=0 is UTF-8 in kannel and not LATIN1 anymore. At the same time if you send
> LATIN1 and expect all chars to arrive on GSM charset then you are wrong. Handset simple
> unable to display chars that are not in GSM03.38 and if you read this function then you will see
> that was intention:
>     /* convert to and the from gsm, so we drop all non GSM chars */
>     charset_utf8_to_gsm(msg->sms.msgdata);
>     charset_gsm_to_utf8(msg->sms.msgdata);
>
> This is consider as bug only on cdma/tdma networks but not on GSM.
>
> If you wish to send chars that are not in GSM03.38 you have to use UCS2 coding but then limited to 70
> "real" chars.
>
> Thanks,
> Alexander Malysh
>
> Am 22.09.2010 um 00:19 schrieb Sanjay Bhandari:
>
>> Hello Alexander
>>
>> Thanks for getting back.
>>
>> I see what you are saying, and I am with you on the fact that
>> alt_charset option is a SMSC option.
>>
>> But in deciding if this is a bug or not, consider the fact that if a
>> user configures alt_charset = ISO-8859-1, and then encodes data using
>> ISO-8859-1 charset (setting coding = 0), he expects the data to be
>> passed through to the SMSC transparently. But what happens is that
>> Kannel replaces all characters in ISO-8859-1 that are not in GSM 03.38
>> by "?". And this is because of the code that I pointed out. If I disable
>> that code, things come out OK.
>>
>> Sanjay
>>
>> On 9/21/2010 6:05 PM, Alexander Malysh wrote:
>>> HI,
>>>
>>> no it's not a bug IMO. in extract_msgdata_part_by_coding we don't convert anything.we just use
>>> gsm03.38 covert function to be able to count message length because equal which charset smsc operator
>>> accept message will be transported via ss7 and there is only gsm03.38 apply (ok may be issue here with cdma/tdma).
>>>
>>> alt_charset is SMSC module option and only tell SMSC module to send message body in this charset.
>>>
>>> Thanks,
>>> Alexander Malysh
>>>
>>> Am 21.09.2010 um 15:10 schrieb Sanjay Bhandari:
>>>
>>>> Can anyone shed light on this?
>>>>
>>>> In file sms.c function extract_msgdata_part_by_coding(), it filters on
>>>> the GSM 03.38 charset. This basically results in characters not in the
>>>> GSM charset to be replaced by "?".
>>>>
>>>> But this happens with no consideration to the alt_charset config setting
>>>> in the "group = smsc". So, basically, if someone sets the alt_charset to
>>>> something other than GSM 03.38, none of the characters not in GSM 03.38
>>>> make it through.
>>>>
>>>> Why is this? Is this not a problem?
>>>>
>>>>
>