Re: unicode problem with sqlbox Revision: 62
Ruben Melikyan <[email protected]> Thu, 4 Jul 2024 09:39:21 +0000
| Newsgroups | gmane.comp.mobile.kannel.user |
|---|---|
| Message-ID | <PAVP195MB2233468ACB1C400AE4026FBDDFDE2@PAVP195MB2233.EURP195.PROD.OUTLOOK.COM> |
--_000_PAVP195MB2233468ACB1C400AE4026FBDDFDE2PAVP195MB2233EURP_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi Dinos, Please paste here your sql query which inserts unicode message to SendTable= . Please provide full query Best regards, Ruben Melikyan ________________________________ From: Konstantinos Liadakis <[email protected]> Sent: Thursday, July 4, 2024 11:44:39 AM To: Ruben Melikyan <[email protected]>; [email protected] <[email protected]> Subject: Re: unicode problem with sqlbox Revision: 62 Hellomy replies below to your questions 1. I can send normal sms with no problem, it is just unicode that i am not = able to send 2. MariaDB [kannel]> show variables like '%char%'; +--------------------------+----------------------------+ | Variable_name | Value | +--------------------------+----------------------------+ | character_set_client | utf8mb4 | | character_set_connection | utf8mb4 | | character_set_database | utf8mb4 | | character_set_filesystem | binary | | character_set_results | utf8mb4 | | character_set_server | utf8mb4 | | character_set_system | utf8mb3 | | character_sets_dir | /usr/share/mysql/charsets/ | +--------------------------+----------------------------+ 8 rows in set (0.002 sec) MariaDB [kannel]> show variables like '%collation%'; +----------------------+--------------------+ | Variable_name | Value | +----------------------+--------------------+ | collation_connection | utf8mb4_general_ci | | collation_database | utf8mb4_general_ci | | collation_server | utf8mb4_general_ci | +----------------------+--------------------+ 3 rows in set (0.001 sec) 3. I am sending message by making an insert to mysql 4. Yes 5. The table is created by sqlbox it self MariaDB [kannel]> select * from sent_sms; +--------+------+----------+--------------+---------+----------------------= ---------------------------------------------------------------------------= ---------------+------------+--------------+---------+------------+------+-= ---------+--------+------+--------+----------+----------+----------+-------= ---+-----------------------------------------------------------------------= ---------------------------------------------------------------------------= ------+------+---------+------+---------+---------+-------+----------------= --------------------------------+----------+------------+ | sql_id | momt | sender | receiver | udhdata | msgdata = = | time | smsc_id | service | account | id | = sms_type | mclass | mwi | coding | compress | validity | deferred | dlr_ma= sk | dlr_url = = | pid | alt_dcs | rpi | charset | boxc_id | binfo | meta_data = | priority | foreign_id | +--------+------+----------+--------------+---------+----------------------= ---------------------------------------------------------------------------= ---------------+------------+--------------+---------+------------+------+-= ---------+--------+------+--------+----------+----------+----------+-------= ---+-----------------------------------------------------------------------= ---------------------------------------------------------------------------= ------+------+---------+------+---------+---------+-------+----------------= --------------------------------+----------+------------+ I installed kannel from svn, and also sqlbox. Today i did make clean in sqlbox and recompiled the one from addons folder,= but i is exactly the same version when doing svn info I am sure if i install an older version everything will work.... Thank you for your efforts Best regards Dinos On 4/7/24 07:24, Ruben Melikyan wrote: Hi Dinos, If so, I need more info: 1. if you send ASCII( not unicode message), what happens? Do you see rec= ord in your table? 2. What are your DB settings ? 3. How do you send message? Please provide detailed description with ex= ample. 4. Did you try to send both ASCII and UNICODE message using SendTable? I= f no, refer to sqlbox user guide "Inserting MT messages by SQL" section and= try if it works. 5. Please also provide your SentTable structure P.S. Assuming that you are using sources from Kannel.org, using current sta= ble version (1.4.5), did not change anything in sources and had compiled sq= lbox using sources located in original Addons folder. Best regards, Ruben Melikyan ________________________________ From: Konstantinos Liadakis <[email protected]><mailto:kliadakis@yuboto.= com> Sent: Thursday, July 4, 2024 2:20:58 AM To: Ruben Melikyan <[email protected]><mailto:[email protected]>; [email protected]<ma= ilto:[email protected]> <[email protected]><mailto:[email protected]> Subject: Re: unicode problem with sqlbox Revision: 62 Hello Ruben, and thank you for your reply. The tables were created automatically from sqlbox like you propose. In fact= i have dropped the tables a few times but no change Regards Dinos On 3/7/24 19:06, Ruben Melikyan wrote: Hi, Obviously you have problem with your mysql database tables. You can leave table creation to sqlbox itself. This is the best way. Just leave database empty and run kannel. If your kannel configured properl= y, sqlbox will create tables automatically. Referring to sqlbox documentation: Database Tables Sqlbox creates it's DB tables on the fly if the tables are not present at t= hat moment. P.S. Kannel=92s sqlbox saving SMS message incliding unicode in encoded form= at. Best regards, Ruben Melikyan From: users <[email protected]><mailto:[email protected]> On = Behalf Of Konstantinos Liadakis Sent: 03.07.2024 17:13 To: [email protected]<mailto:[email protected]> Subject: unicode problem with sqlbox Revision: 62 Hello, I am trying to send unicode using sqlbox but i am unable. I am getting the = following error 2024-07-03 12:53:37 [46054] [1] ERROR: MYSQL: You have an error in your SQL= syntax; check the manual that corresponds to your MariaDB server version f= or the right syntax to use near '' at line 1 2024-07-03 12:53:37 [46054] [1] ERROR: MYSQL: You have an error in your SQL= syntax; check the manual that corresponds to your MariaDB server version f= or the right syntax to use near ')' at line 1 2024-07-03 12:53:37 [46054] [1] ERROR: Could not charset process message, d= ropping it! I am using Kannel bearerbox version `svn-r5335M'. Build `Apr 21 2024 15:23:40', compil= er `11.4.0'. System Linux, release 5.15.0-113-generic, version #123-Ubuntu = SMP Mon Jun 10 08:16:17 UTC 2024, machine x86_64. Hostname kannel3.srv.exam= ple.com, IP 1.1.1.1 Libxml version 2.9.13. Using OpenSSL 3.0.2 15 Mar 2022.= Compiled with MySQL 8.0.36, using MySQL 8.0.37. Using hiredis API 0.14.1 U= sing native malloc. The sqlbox version below does not work with unicode Working Copy Root Path: /usr/local/src/kannel/sqlbox URL: https://svn.kannel.org/sqlbox/trunk Relative URL: ^/trunk Repository Root: https://svn.kannel.org/sqlbox Revision: 62 Node Kind: directory Schedule: normal Last Changed Author: stolj Last Changed Rev: 62 Last Changed Date: 2023-03-09 11:22:07 +0000 (Thu, 09 Mar 2023) I am having an older instance where unicode from sqlbox works with no probl= em Path: . Working Copy Root Path: /usr/local/src/kannel/sqlbox URL: https://svn.kannel.org/sqlbox/trunk Relative URL: ^/trunk Repository Root: https://svn.kannel.org/sqlbox Revision: 58 Node Kind: directory Schedule: normal Last Changed Author: rkluwen Last Changed Rev: 58 Last Changed Date: 2017-07-12 23:38:55 +0300 (Wed, 12 Jul 2017) I am using coding=3D2 and charset utf-8 and messages are url encoded. In fa= ct same query that works on old sqlbox does not work on the newer one givin= g the charset error above I tried applying the sqlbox patch but nothing changed. Maybe i am missing s= omething? When should the patch be applied? Thank you in advance for your help regards Dinos --_000_PAVP195MB2233468ACB1C400AE4026FBDDFDE2PAVP195MB2233EURP_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> </head> <body> <div> <div dir=3D"ltr"> <div dir=3D"ltr">Hi Dinos,</div> <div dir=3D"ltr">Please paste here your sql query which inserts unicode mes= sage to SendTable. </div> <div dir=3D"ltr"><br> </div> <div dir=3D"ltr">Please provide full query</div> </div> </div> <div id=3D"ms-outlook-mobile-signature"> <div><br> </div> <div dir=3D"ltr"> <div dir=3D"ltr">Best regards,</div> <div dir=3D"ltr">Ruben Melikyan</div> </div> </div> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st= yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Konstantinos Liadakis= <[email protected]><br> <b>Sent:</b> Thursday, July 4, 2024 11:44:39 AM<br> <b>To:</b> Ruben Melikyan <[email protected]>; [email protected] <users@= kannel.org><br> <b>Subject:</b> Re: unicode problem with sqlbox Revision: 62</font> <div> </div> </div> <div> <p>Hellomy replies below to your questions</p> <p>1. I can send normal sms with no problem, it is just unicode that i am n= ot able to send</p> <p>2. MariaDB [kannel]> show variables like '%char%';<br> +--------------------------+----------------------------+<br> | Variable_name = | Value &= nbsp; |<br> +--------------------------+----------------------------+<br> | character_set_client | utf8mb4 &= nbsp; &nbs= p; |<br> | character_set_connection | utf8mb4 &nb= sp; = |<br> | character_set_database | utf8mb4  = ; &n= bsp; |<br> | character_set_filesystem | binary &nbs= p; &= nbsp; |<br> | character_set_results | utf8mb4 = &nb= sp; |<br> | character_set_server | utf8mb4 &= nbsp; &nbs= p; |<br> | character_set_system | utf8mb3 &= nbsp; &nbs= p; |<br> | character_sets_dir | /usr/share/mysql= /charsets/ |<br> +--------------------------+----------------------------+<br> 8 rows in set (0.002 sec)<br> <br> MariaDB [kannel]> show variables like '%collation%';<br> +----------------------+--------------------+<br> | Variable_name | Value &nbs= p; |<br> +----------------------+--------------------+<br> | collation_connection | utf8mb4_general_ci |<br> | collation_database | utf8mb4_general_ci |<br> | collation_server | utf8mb4_general_ci |<br> +----------------------+--------------------+<br> 3 rows in set (0.001 sec)</p> <p><br> </p> <p>3. I am sending message by making an insert to mysql</p> <p>4. Yes</p> <p>5. The table is created by sqlbox it self</p> <p>MariaDB [kannel]> select * from sent_sms;<br> +--------+------+----------+--------------+---------+----------------------= ---------------------------------------------------------------------------= ---------------+------------+--------------+---------+------------+------+-= ---------+--------+------+--------+----------+----------+----------+-------= ---+-----------------------------------------------------------------------= ---------------------------------------------------------------------------= ------+------+---------+------+---------+---------+-------+----------------= --------------------------------+----------+------------+<br> | sql_id | momt | sender | receiver | u= dhdata | msgdata  = ; &n= bsp;  = ; &n= bsp;  = ; &n= bsp;  = ; &n= bsp; | time &nbs= p; | smsc_id | service | account &= nbsp; | id | sms_type | mclass | mwi | coding | compress | validity | deferred | dlr_mask | dlr_url &nb= sp; = &nb= sp; = &nb= sp; = &nb= sp; = &nb= sp; = &nb= sp; = | pid | alt_dcs | rpi | charset | boxc_id | binfo = | meta_data &nbs= p; &= nbsp; &nbs= p; | priority | foreign_id |<br> +--------+------+----------+--------------+---------+----------------------= ---------------------------------------------------------------------------= ---------------+------------+--------------+---------+------------+------+-= ---------+--------+------+--------+----------+----------+----------+-------= ---+-----------------------------------------------------------------------= ---------------------------------------------------------------------------= ------+------+---------+------+---------+---------+-------+----------------= --------------------------------+----------+------------+<br> </p> <p>I installed kannel from svn, and also sqlbox.</p> <p>Today i did make clean in sqlbox and recompiled the one from addons fold= er, but i is exactly the same version when doing svn info</p> <p>I am sure if i install an older version everything will work....</p> <p>Thank you for your efforts</p> <p>Best regards</p> <p>Dinos<br> </p> <div class=3D"x_moz-cite-prefix">On 4/7/24 07:24, Ruben Melikyan wrote:<br> </div> <blockquote type=3D"cite"> <div> <div dir=3D"ltr"> <div dir=3D"ltr">Hi Dinos,</div> <div dir=3D"ltr">If so, I need more info:</div> <div dir=3D"ltr"> <ol data-editing-info=3D"{"orderedStyleType":1,"unorderedSty= leType":1}" style=3D"margin-top:0px; margin-bottom:0px"> <li style=3D"list-style-type:"1. ""><span>if you send ASCII( not = unicode message), what happens? Do you see record in your table?</span></li= ><li style=3D"list-style-type:"2. ""><span>What are your DB setti= ngs ?<span></span></span></li><li style=3D"list-style-type:"3. ""= ><span>How do you send message? Please provide detailed description w= ith example.<span></span></span></li><li style=3D"list-style-type:"4. = ""><span>Did you try to send both ASCII and UNICODE message using Send= Table? If no, refer to sqlbox user guide "Inserting MT messages by SQL= " section and try if it works.</span></li><li style=3D"list-style-type= :"5. "; font-size:inherit"><span style=3D"font-size:inherit">Plea= se also provide your SentTable structure</span></li></ol> </div> <div dir=3D"ltr"><span></span></div> <div dir=3D"ltr"><br> </div> <div dir=3D"ltr">P.S. Assuming that you are using sources from Kannel.org, = using current stable version (1.4.5), did not change anything in sources an= d had compiled sqlbox using sources located in original Addons folder.</div= > </div> </div> <div id=3D"x_ms-outlook-mobile-signature"> <div><br> </div> <div dir=3D"ltr"> <div dir=3D"ltr">Best regards,</div> <div dir=3D"ltr">Ruben Melikyan</div> </div> </div> <hr tabindex=3D"-1" style=3D"display:inline-block; width:98%"> <div id=3D"x_divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" = color=3D"#000000" style=3D"font-size:11pt"><b>From:</b> Konstantinos Liadak= is <a class=3D"x_moz-txt-link-rfc2396E" href=3D"mailto:[email protected]">&= lt;[email protected]></a><br> <b>Sent:</b> Thursday, July 4, 2024 2:20:58 AM<br> <b>To:</b> Ruben Melikyan <a class=3D"x_moz-txt-link-rfc2396E" href=3D"mail= to:[email protected]"> <[email protected]></a>; <a class=3D"x_moz-txt-link-abbreviated" href=3D"m= ailto:[email protected]"> [email protected]</a> <a class=3D"x_moz-txt-link-rfc2396E" href=3D"mailto:us= [email protected]"> <[email protected]></a><br> <b>Subject:</b> Re: unicode problem with sqlbox Revision: 62</font> <div> </div> </div> <div> <p>Hello Ruben, and thank you for your reply.</p> <p>The tables were created automatically from sqlbox like you propose. In f= act i have dropped the tables a few times but no change</p> <p>Regards</p> <p>Dinos<br> </p> <div class=3D"x_x_moz-cite-prefix">On 3/7/24 19:06, Ruben Melikyan wrote:<b= r> </div> <blockquote type=3D"cite"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style> <!-- @font-face {font-family:"Cambria Math"} @font-face {font-family:Calibri} p.x_x_MsoNormal, li.x_x_MsoNormal, div.x_x_MsoNormal {margin:0in; font-size:11.0pt; font-family:"Calibri",sans-serif} a:link, span.x_x_MsoHyperlink {color:blue; text-decoration:underline} span.x_x_EmailStyle21 {font-family:"Calibri",sans-serif; color:windowtext} .x_x_MsoChpDefault {font-size:10.0pt} --> </style> <div class=3D"x_x_WordSection1"> <p class=3D"x_x_MsoNormal"><a name=3D"x_x__Hlk120024655"><span lang=3D"EN-G= B">Hi,</span></a></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB">Obviously = you have problem with your mysql database tables.</span></span></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB">You can le= ave table creation to sqlbox itself. This is the best way.</span></span></p= > <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB"> </sp= an></span></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB">Just leave= database empty and run kannel. If your kannel configured properly, sqlbox = will create tables automatically.</span></span></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB"> </sp= an></span></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB">Referring = to sqlbox documentation:</span></span></p> <p class=3D"x_x_MsoNormal" style=3D"background:white"><span style=3D""><a n= ame=3D"x_x_AEN473"><b><span style=3D"font-size:13.5pt; font-family:"Ti= mes New Roman",serif; color:black">Database Tables</span></b></a></spa= n><span style=3D""><b><span style=3D"font-size:13.5pt; font-family:"Ti= mes New Roman",serif; color:black"></span></b></span></p> <p class=3D"x_x_MsoNormal" style=3D"background:white"><span style=3D""><spa= n style=3D"font-size:13.5pt; font-family:"Times New Roman",serif;= color:black">Sqlbox creates it's DB tables on the fly if the tables are no= t present at that moment.</span></span></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB"> </sp= an></span></p> <p class=3D"x_x_MsoNormal"><span style=3D""><span lang=3D"EN-GB">P.S. Kanne= l=92s sqlbox saving SMS message incliding unicode in encoded format.</span>= </span></p> <span style=3D""></span> <div> <p class=3D"x_x_MsoNormal" style=3D""><b><span style=3D"font-size:12.0pt; f= ont-family:"Times New Roman",serif">Best regards,</span></b><span= style=3D"font-size:12.0pt; font-family:"Times New Roman",serif"> <br> <br> <b>Ruben Melikyan</b></span></p> </div> <p class=3D"x_x_MsoNormal"><span lang=3D"EN-GB"> </span></p> <div> <div style=3D"border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0i= n 0in 0in"> <p class=3D"x_x_MsoNormal"><b>From:</b> users <a class=3D"x_x_moz-txt-link-= rfc2396E" href=3D"mailto:[email protected]"> <[email protected]></a> <b>On Behalf Of </b>Konstantinos Liada= kis<br> <b>Sent:</b> 03.07.2024 17:13<br> <b>To:</b> <a class=3D"x_x_moz-txt-link-abbreviated x_moz-txt-link-freetext= " href=3D"mailto:[email protected]"> [email protected]</a><br> <b>Subject:</b> unicode problem with sqlbox Revision: 62</p> </div> </div> <p class=3D"x_x_MsoNormal"> </p> <p class=3D"x_x_MsoNormal">Hello, </p> <div> <p>I am trying to send unicode using sqlbox but i am unable. I am getting t= he following error</p> <p>2024-07-03 12:53:37 [46054] [1] ERROR: MYSQL: You have an error in your = SQL syntax; check the manual that corresponds to your MariaDB server versio= n for the right syntax to use near '' at line 1<br> 2024-07-03 12:53:37 [46054] [1] ERROR: MYSQL: You have an error in your SQL= syntax; check the manual that corresponds to your MariaDB server version f= or the right syntax to use near ')' at line 1<br> 2024-07-03 12:53:37 [46054] [1] ERROR: Could not charset process message, d= ropping it!</p> <p>I am using </p> <p>Kannel bearerbox version `svn-r5335M'. Build `Apr 21 2024 15:23:40', com= piler `11.4.0'. System Linux, release 5.15.0-113-generic, version #123-Ubun= tu SMP Mon Jun 10 08:16:17 UTC 2024, machine x86_64. Hostname kannel3.srv.e= xample.com, IP 1.1.1.1 Libxml version 2.9.13. Using OpenSSL 3.0.2 15 Mar 2022. Compiled with MySQL 8.0.36, using= MySQL 8.0.37. Using hiredis API 0.14.1 Using native malloc.</p> <p>The sqlbox version below <b>does not work with unicode</b></p> <p>Working Copy Root Path: /usr/local/src/kannel/sqlbox<br> URL: <a href=3D"https://svn.kannel.org/sqlbox/trunk" class=3D"x_x_moz-txt-l= ink-freetext x_moz-txt-link-freetext"> https://svn.kannel.org/sqlbox/trunk</a><br> Relative URL: ^/trunk<br> Repository Root: <a href=3D"https://svn.kannel.org/sqlbox" class=3D"x_x_moz= -txt-link-freetext x_moz-txt-link-freetext"> https://svn.kannel.org/sqlbox</a><br> Revision: 62<br> Node Kind: directory<br> Schedule: normal<br> Last Changed Author: stolj<br> Last Changed Rev: 62<br> Last Changed Date: 2023-03-09 11:22:07 +0000 (Thu, 09 Mar 2023)</p> <p> </p> <p>I am having an older instance where unicode from sqlbox <b>works</b> wit= h no problem</p> <p>Path: .<br> Working Copy Root Path: /usr/local/src/kannel/sqlbox<br> URL: <a href=3D"https://svn.kannel.org/sqlbox/trunk" class=3D"x_x_moz-txt-l= ink-freetext x_moz-txt-link-freetext"> https://svn.kannel.org/sqlbox/trunk</a><br> Relative URL: ^/trunk<br> Repository Root: <a href=3D"https://svn.kannel.org/sqlbox" class=3D"x_x_moz= -txt-link-freetext x_moz-txt-link-freetext"> https://svn.kannel.org/sqlbox</a><br> Revision: 58<br> Node Kind: directory<br> Schedule: normal<br> Last Changed Author: rkluwen<br> Last Changed Rev: 58<br> Last Changed Date: 2017-07-12 23:38:55 +0300 (Wed, 12 Jul 2017)</p> <p>I am using coding=3D2 and charset utf-8 and messages are url encoded. In= fact same query that works on old sqlbox does not work on the newer one gi= ving the charset error above</p> <p>I tried applying the sqlbox patch but nothing changed. Maybe i am missin= g something? When should the patch be applied?</p> <p>Thank you in advance for your help </p> <p>regards</p> <p>Dinos</p> </div> </div> </blockquote> </div> </blockquote> </div> </body> </html> --_000_PAVP195MB2233468ACB1C400AE4026FBDDFDE2PAVP195MB2233EURP_--