TLV does not respect UC2 flag

"[email protected]" <[email protected]> Mon, 25 May 2020 17:23:24 -0400
Newsgroups	gmane.comp.mobile.kannel.user,gmane.comp.mobile.kannel.devel
Message-ID	<[email protected]>
This is a multi-part message in MIME format.
--------------209F6664CF93E3529E9F687C
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

**
&coding=3D2&charset=3Dutf8&text=3DUnicode+Greek+=CE=8C=CE=BB=CE=B1 =CE=9A=
=CE=B1=CE=BB=CE=AC

 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0 sho=
rt_message:
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0 Octet string at
0x7fe9e8006c80:
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 len:=C2=A0 44
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 size: 45
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 immutable: 0
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 data: 00 55 00 6e 00 69
00 63 00 6f 00 64 00 65 00 20=C2=A0=C2=A0 .U.n.i.c.o.d.e.
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 data: 00 47 00 72 00 65
00 65 00 6b 00 20 03 8c 03 bb=C2=A0=C2=A0 .G.r.e.e.k. ....
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 data: 03 b1 00 20 03 9a
03 b1 03 bb 03 ac=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... ........
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=
=A0 Octet string dump ends.
 =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG: SMPP PDU dump e=
nds.

When submitting unicode message in TLV 0x424 field and coding=3D2 and
char-set=3Dutf-8, message is not converted to ucs2 and delivered as Chines=
e.


&coding=3D2&charset=3Dutf8&text=3D&meta-data=3D%3Fsmpp%3Fmessage-payload%3=
D%=3DUnicode+Greek+=CE=8C=CE=BB=CE=B1
=CE=9A=CE=B1=CE=BB=CE=AC

[6] DEBUG:=C2=A0=C2=A0=C2=A0 Octet string at 0x7fe9e8006e10:
2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 len:=
=C2=A0 29
2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size:=
 30
2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 immut=
able: 0
2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 data:=
 55 6e 69 63 6f 64 65
20 47 72 65 65 6b 20 ce 8c=C2=A0=C2=A0 Unicode Greek ..
2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 data:=
 ce bb ce b1 20 ce 9a
ce b1 ce bb ce ac=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 .... ........
2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0 Octet string dump=
 ends.
2020-05-25 21:05:11 [24722] [6] DEBUG: SMPP PDU dump ends.

is it a bug or we have encode meta-data differently somehow?


--------------209F6664CF93E3529E9F687C
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html>
  <head>
    <meta http-equiv=3D"content-type" content=3D"text/html; charset=3DUTF-=
8">
  </head>
  <body>
    <dl>
      <dt><tt><strong></strong></tt> <br>
        &amp;coding=3D2&amp;charset=3Dutf8&amp;text=3DUnicode+Greek+=CE=8C=
=CE=BB=CE=B1 =CE=9A=CE=B1=CE=BB=CE=AC<br>
        <br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0 short_message:<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0 Octet string at
        0x7fe9e8006c80:<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 len:=C2=A0 44<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 size: 45<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 immutable: 0<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 data: 00 55 00
        6e 00 69 00 63 00 6f 00 64 00 65 00 20=C2=A0=C2=A0 .U.n.i.c.o.d.e.=
<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 data: 00 47 00
        72 00 65 00 65 00 6b 00 20 03 8c 03 bb=C2=A0=C2=A0 .G.r.e.e.k. ...=
.<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 data: 03 b1 00
        20 03 9a 03 b1 03 bb 03 ac=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... ........<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG:=C2=A0=C2=
=A0=C2=A0 Octet string dump
        ends.<br>
        =C2=A0=C2=A0=C2=A0 2020-05-25 20:37:03 [24722] [6] DEBUG: SMPP PDU=
 dump ends.<br>
        <br>
        When submitting unicode message in TLV 0x424 field and coding=3D2
        and char-set=3Dutf-8, message is not converted to ucs2 and
        delivered as Chinese.<br>
        <br>
        <br>
&amp;coding=3D2&amp;charset=3Dutf8&amp;text=3D&amp;meta-data=3D%3Fsmpp%3Fm=
essage-payload%3D%=3DUnicode+Greek+=CE=8C=CE=BB=CE=B1
        =CE=9A=CE=B1=CE=BB=CE=AC <br>
        <br>
        [6] DEBUG:=C2=A0=C2=A0=C2=A0 Octet string at 0x7fe9e8006e10:<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 len:=C2=A0 29<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 size: 30<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 immutable: 0<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 data: 55 6e 69 63 6f
        64 65 20 47 72 65 65 6b 20 ce 8c=C2=A0=C2=A0 Unicode Greek ..<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 data: ce bb ce b1 20
        ce 9a ce b1 ce bb ce ac=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 .... ........<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG:=C2=A0=C2=A0=C2=A0 Octet str=
ing dump
        ends.<br>
        2020-05-25 21:05:11 [24722] [6] DEBUG: SMPP PDU dump ends.</dt>
    </dl>
    <p>is it a bug or we have encode meta-data differently somehow?<br>
    </p>
  </body>
</html>

--------------209F6664CF93E3529E9F687C--