Re: Can't convert sjis&ujis half-width katakana correctly

   Hello, Yoshinori!

You are right, this is a bug, and your fix looks fine.
At least for SJIS (I haven't studied for UJIS yet).

I posted it into our bug system and now working on it.

http://bugs.mysql.com/bug.php?id=3290

Thank you very much for so defailed report.


Matsunobu, Yoshinori wrote:
> Hello.
> 
> Now I'm evaluating the functionality of Japanese character sets
> conversion of MySQL 5.0.0 alpha.
> 
> When I inserted Japanese half-width katakana into tables and selected
> from them, 
> the following problems occurred (probably bugs).
> 
> 1. Sjis half-width katakana characters were converted to wrong
> characters.
> 2. Ujis half-width katakana characters were converted to wrong
> characters.
> 
> Environment:
> 	Version: MySQL 5.0.0 alpha (source code install)
> 	Platform: RedHat Linux 9
> 	Install option :./configure --prefix=/usr/local/mysql50
> --with-unix-socket-path=/usr/local/mysql50/tmp/mysql.sock
> --with-extra-charsets=all --with-charset=sjis 
> 
> How-To-Repeat:
> 	1.sjis problem
> 	create database db1 character set utf8;
> 	CREATE TABLE table1(
> 	col1 varchar(100)
> 	)  default character set utf8;
> 	
> 	set names sjis;
> 	source /path/to/sjis-half-width-katakana-format-file;
> 	select * from table1;
> 
> 	2.ujis problem
> 	truncate table table1;
> 	set names ujis;
> 	source /path/to/ujis-half-width-katakana-format-file;
> 	select * from table1;
> 
> 
> Fix:
> I read MySQL source code,and I found there are some kinds of bugs.
> I fixed the following, tested, and confirmed that all the above problems
> were solved(in my environment).
> 
> 1.ctype-sjis.c
> 1-1.function my_mb_wc_sjis
> 
> current:
> ....
>   if (hi<0x80)
>   {
>     pwc[0]=hi;
>     return 1;
>   }
>   if (s+2>e)
>     return MY_CS_TOOFEW(0);
> ....
> 
> I fixed:
> ....
>   if (hi<0x80)
>   {
>     pwc[0]=hi;
>     return 1;
>   }
> 
>   //I added
>   if((hi>=0xA1)&&(hi<=0xDF))
>   {
>       pwc[0]=func_sjis_uni_onechar(hi);
>       return 1;
>   }
> 
>   if (s+2>e)
>     return MY_CS_TOOFEW(0);
> ....
> 
> 
> 1-2.function my_mb_wb_sjis
> 
> current:
> ....
>   if (!(code=func_uni_sjis_onechar(wc)))
>     return MY_CS_ILUNI;
>   if (s+2>e)
>     return MY_CS_TOOSMALL;
>   
>   s[0]=code>>8;
>   s[1]=code&0xFF;
>   return 2;
> ....
> 
> I fixed:
> ....
>   if (!(code=func_uni_sjis_onechar(wc)))
>     return MY_CS_ILUNI;
> 
>   //I added
>   if((code>=0xA1)&&(code<=0xDF))
>   {
>       s[0]=code;
>       return 1;
>   }
> 
>   if (s+2>e)
>     return MY_CS_TOOSMALL;
>   
>   s[0]=code>>8;
>   s[1]=code&0xFF;
>   return 2;
> ....
> 
> 
> 
> 2.ctype-ujis.c  
> function my_wc_mb_euc_jp
> 
> current:
> ....
>   ret=my_wc_mb_jisx0201(c,wc,buf,buf+2);
>   if (ret==1)
>   {
>     if (s+1>e)
>       return MY_CS_TOOSMALL;
>       
>     s[0]=0x8E;
>     s[1]=buf[0];
>     return 1;
>   }
> ....
> 
> I fixed:
> ....
>   ret=my_wc_mb_jisx0201(c,wc,buf,buf+2);
>   if (ret==1)
>   {
>     if (s+1>e)
>       return MY_CS_TOOSMALL;
>       
>     s[0]=0x8E;
>     s[1]=buf[0];
>     return 2;     //Because ujis half-width katakana is 2 byte.
>   }
> ....
> 
> 
> Best Regards,
> 
> Matsunobu Yoshinori.
> 


-- 
For technical support contracts, visit https://order.mysql.com/
    __  ___     ___ ____  __
   /  |/  /_ __/ __/ __ \/ /    Mr. Alexander Barkov <[email protected]>
  / /|_/ / // /\ \/ /_/ / /__   MySQL AB, Full-Time Developer
/_/  /_/\_, /___/\___\_\___/   Izhevsk, Russia
        <___/   www.mysql.com   +7-912-856-80-21


-- 
MySQL Bugs Mailing List
For list archives: http://lists.mysql.com/bugs
To unsubscribe:    http://lists.mysql.com/[email protected]