Re: [PATCH] handling of non-latin GSM subset in opensmppbox

That could also be the issue that Mike is facing with sqlbox....

Nikos
----- Original Message ----- 
From: "Nikos Balkanas" <[email protected]>
To: "'Rene Kluwen'" <[email protected]>; "'Nii Ako Ampa-Sowa'" 
<[email protected]>; <[email protected]>
Sent: Sunday, May 08, 2011 2:32 AM
Subject: Re: [PATCH] handling of non-latin GSM subset in opensmppbox


> Registered data_coding values for SMPP are:
>
> Bits 7 6 5 4 3 2 1 0 Meaning Notes
>        0 0 0 0 0 0 0 0 SMSC Default Alphabet
>        0 0 0 0 0 0 0 1 IA5 (CCITT T.50)/ASCII (ANSI X3.4) b
>        0 0 0 0 0 0 1 0 Octet unspecified (8-bit binary) b
>        0 0 0 0 0 0 1 1 Latin 1 (ISO-8859-1) b
>        0 0 0 0 0 1 0 0 Octet unspecified (8-bit binary) a
>        0 0 0 0 0 1 0 1 JIS (X 0208-1990) b
>        0 0 0 0 0 1 1 0 Cyrllic (ISO-8859-5) b
>        0 0 0 0 0 1 1 1 Latin/Hebrew (ISO-8859-8) b
>        0 0 0 0 1 0 0 0 UCS2 (ISO/IEC-10646) a
>        0 0 0 0 1 0 0 1 Pictogram Encoding b
>        0 0 0 0 1 0 1 0 ISO-2022-JP (Music Codes) b
>        0 0 0 0 1 0 1 1 reserved
>        0 0 0 0 1 1 0 0 reserved
>        0 0 0 0 1 1 0 1 Extended Kanji JIS(X 0212-1990) b
>        0 0 0 0 1 1 1 0 KS C 5601 b
>        0 0 0 0 1 1 1 1 reserved
>        1 0 1 1 1 1 1 1 reserved
>        1 1 0 0 x x x x GSM MWI control - see [GSM 03.38] d
>        1 1 0 1 x x x x GSM MWI control - see [GSM 03.38] d
>        1 1 1 0 x x x x reserved
>        1 1 1 1 x x x x GSM message class control - see [GSM 03.38] e
>
> With default data_coding (0) you could have a slew of charsets. That's the 
> only way to pass iso-8859-7 (Greek). GSM and UCS2 should be determined 
> from their values. But between the different iso flavors, it is 
> impossible. It would probably be better to be configurable per ESME 
> connection, but it doesn't need to be so. If the wrong iso is assumed, as 
> long as the alt-charset in bearerbox is the same, it will be deconverted 
> correctly, even if the intermediate UTF-8 is wrong. Of course, a 
> full-spectrum iso like iso-8859-4 should be chosen, that contains all iso 
> values.
>
> BR,
> Nikos
> ----- Original Message ----- 
> From: "Rene Kluwen" <[email protected]>
> To: "'Rene Kluwen'" <[email protected]>; "'Nikos Balkanas'" 
> <[email protected]>; "'Nii Ako Ampa-Sowa'" <[email protected]>; 
> <[email protected]>
> Sent: Sunday, May 08, 2011 2:09 AM
> Subject: RE: [PATCH] handling of non-latin GSM subset in opensmppbox
>
>
>> Oops... Nikos... I replied on a wrong thread. This was not about sqlbox.
>> My real question: What type of charset do the SMPP specs dictate? Do you
>> think in opensmppbox there should be a configuration value that indicates 
>> a
>> charset?
>>
>> == Rene
>>
>> -----Original Message-----
>> From: Rene Kluwen [mailto:[email protected]]
>> Sent: Sunday, 08 May, 2011 01:07
>> To: 'Nikos Balkanas'; 'Nii Ako Ampa-Sowa'; '[email protected]'
>> Subject: RE: [PATCH] handling of non-latin GSM subset in opensmppbox
>>
>> It's what I thought also.
>> There is a charset field in the send_sms table. Nikos, do you think that 
>> one
>> should be accordingly?
>>
>> == Rene
>>
>> -----Original Message-----
>> From: Nikos Balkanas [mailto:[email protected]]
>> Sent: Sunday, 08 May, 2011 01:02
>> To: Rene Kluwen; 'Nii Ako Ampa-Sowa'; [email protected]
>> Subject: Re: [PATCH] handling of non-latin GSM subset in opensmppbox
>>
>> Yes, but that is only half the story. What about if the input is not GSM,
>> but an iso-*? Shouldn't it be converted to UTF-8, too? The standard also 
>> for
>>
>> UCS-2 is UTF-16BE, not left as is.
>>
>> BR,
>> Nikos
>> ----- Original Message ----- 
>> From: Rene Kluwen
>> To: 'Nii Ako Ampa-Sowa' ; [email protected]
>> Sent: Saturday, May 07, 2011 5:16 PM
>> Subject: RE: [PATCH] handling of non-latin GSM subset in opensmppbox
>>
>>
>> I think this patch makes sense. It's an internal coding anyway. and it
>> should be utf-8 instead of latin1.
>> If nobody objects, I will commit during the week.
>>
>> == Rene
>>
>> From: [email protected] [mailto:[email protected]] On 
>> Behalf
>> Of Nii Ako Ampa-Sowa
>> Sent: Friday, 06 May, 2011 23:04
>> To: [email protected]
>> Subject: [PATCH] handling of non-latin GSM subset in opensmppbox
>>
>> Hi,
>>
>> opensmppbox doesn't seem to handle the non-Latin subset of the GSM 
>> character
>>
>> set very well. This affects the Euro sign as well as Greek characters.
>> Attached is a patch that switches from using the Latin1 conversion 
>> routine
>> to the UTF-8 version.
>>
>> Nii
>>
>>
>>
>