Re: [PATCH] handling of non-latin GSM subset in opensmppbox

Registered data_coding values for SMPP are:

Bits 7 6 5 4 3 2 1 0 Meaning Notes
        0 0 0 0 0 0 0 0 SMSC Default Alphabet
        0 0 0 0 0 0 0 1 IA5 (CCITT T.50)/ASCII (ANSI X3.4) b
        0 0 0 0 0 0 1 0 Octet unspecified (8-bit binary) b
        0 0 0 0 0 0 1 1 Latin 1 (ISO-8859-1) b
        0 0 0 0 0 1 0 0 Octet unspecified (8-bit binary) a
        0 0 0 0 0 1 0 1 JIS (X 0208-1990) b
        0 0 0 0 0 1 1 0 Cyrllic (ISO-8859-5) b
        0 0 0 0 0 1 1 1 Latin/Hebrew (ISO-8859-8) b
        0 0 0 0 1 0 0 0 UCS2 (ISO/IEC-10646) a
        0 0 0 0 1 0 0 1 Pictogram Encoding b
        0 0 0 0 1 0 1 0 ISO-2022-JP (Music Codes) b
        0 0 0 0 1 0 1 1 reserved
        0 0 0 0 1 1 0 0 reserved
        0 0 0 0 1 1 0 1 Extended Kanji JIS(X 0212-1990) b
        0 0 0 0 1 1 1 0 KS C 5601 b
        0 0 0 0 1 1 1 1 reserved
        1 0 1 1 1 1 1 1 reserved
        1 1 0 0 x x x x GSM MWI control - see [GSM 03.38] d
        1 1 0 1 x x x x GSM MWI control - see [GSM 03.38] d
        1 1 1 0 x x x x reserved
        1 1 1 1 x x x x GSM message class control - see [GSM 03.38] e

With default data_coding (0) you could have a slew of charsets. That's the 
only way to pass iso-8859-7 (Greek). GSM and UCS2 should be determined from 
their values. But between the different iso flavors, it is impossible. It 
would probably be better to be configurable per ESME connection, but it 
doesn't need to be so. If the wrong iso is assumed, as long as the 
alt-charset in bearerbox is the same, it will be deconverted correctly, even 
if the intermediate UTF-8 is wrong. Of course, a full-spectrum iso like 
iso-8859-4 should be chosen, that contains all iso values.

BR,
Nikos
----- Original Message ----- 
From: "Rene Kluwen" <[email protected]>
To: "'Rene Kluwen'" <[email protected]>; "'Nikos Balkanas'" 
<[email protected]>; "'Nii Ako Ampa-Sowa'" <[email protected]>; 
<[email protected]>
Sent: Sunday, May 08, 2011 2:09 AM
Subject: RE: [PATCH] handling of non-latin GSM subset in opensmppbox


> Oops... Nikos... I replied on a wrong thread. This was not about sqlbox.
> My real question: What type of charset do the SMPP specs dictate? Do you
> think in opensmppbox there should be a configuration value that indicates 
> a
> charset?
>
> == Rene
>
> -----Original Message-----
> From: Rene Kluwen [mailto:[email protected]]
> Sent: Sunday, 08 May, 2011 01:07
> To: 'Nikos Balkanas'; 'Nii Ako Ampa-Sowa'; '[email protected]'
> Subject: RE: [PATCH] handling of non-latin GSM subset in opensmppbox
>
> It's what I thought also.
> There is a charset field in the send_sms table. Nikos, do you think that 
> one
> should be accordingly?
>
> == Rene
>
> -----Original Message-----
> From: Nikos Balkanas [mailto:[email protected]]
> Sent: Sunday, 08 May, 2011 01:02
> To: Rene Kluwen; 'Nii Ako Ampa-Sowa'; [email protected]
> Subject: Re: [PATCH] handling of non-latin GSM subset in opensmppbox
>
> Yes, but that is only half the story. What about if the input is not GSM,
> but an iso-*? Shouldn't it be converted to UTF-8, too? The standard also 
> for
>
> UCS-2 is UTF-16BE, not left as is.
>
> BR,
> Nikos
> ----- Original Message ----- 
> From: Rene Kluwen
> To: 'Nii Ako Ampa-Sowa' ; [email protected]
> Sent: Saturday, May 07, 2011 5:16 PM
> Subject: RE: [PATCH] handling of non-latin GSM subset in opensmppbox
>
>
> I think this patch makes sense. It's an internal coding anyway. and it
> should be utf-8 instead of latin1.
> If nobody objects, I will commit during the week.
>
> == Rene
>
> From: [email protected] [mailto:[email protected]] On Behalf
> Of Nii Ako Ampa-Sowa
> Sent: Friday, 06 May, 2011 23:04
> To: [email protected]
> Subject: [PATCH] handling of non-latin GSM subset in opensmppbox
>
> Hi,
>
> opensmppbox doesn't seem to handle the non-Latin subset of the GSM 
> character
>
> set very well. This affects the Euro sign as well as Greek characters.
> Attached is a patch that switches from using the Latin1 conversion routine
> to the UTF-8 version.
>
> Nii
>
>
>