Re: unicode

Laura,

Just some remarks...

With SQL Server, you must use NCHAR or NVARCHAR to store UNICODE chars.
SQL Server uses UCS-2 on the server side, not UTF-8... However FreeTDS
will support UCS-2/UTF-8 conversion (according to last discussions).

The encoding SQL_Latin1_General_CP1_CI_AS is for CHAR/VARCHAR columns.

What do you use on the Solaris machine to display the Jap text file?

What is the encoding of that text file?

If it's UTF-8, and you use for example an xterm with vi, you need to:

A- Define LANG or LC_ALL to something like en_US.utf8 or ja_JP.utf8.

B- Make sure the xterm is configured to display the correct code set.

Seb

Murray, Laura (US) wrote:
> Hi
> Hello - 
> 
> Our OS is Solaris 10.
> Encoding within the Oracle database is UTF-8
> Encoding of MS SQL Server is: SQL_Latin1_General_CP1_CI_AS
> 
> I tried in vain to do the tsql test.  The Japanese characters do not
> display at all in the Solaris environment (i.e. in a text file on the
> server).  I xferred a small file from my laptop (where the Japanese
> characters appeared properly) to Solaris and on Solaris the Japanese
> characters are garbled.
> However, within the Oracle Database (hosted by this same Solaris system)
> the Japanese characters appear properly.
> 
> I am also working with Oracle Support on this problem, but no luck yet.
> 
> --Laurie
> 
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of James K. Lowden
> Sent: Wednesday, July 30, 2008 12:41 PM
> To: FreeTDS Development Group
> Subject: Re: [freetds] unicode
> 
> Murray, Laura (US) wrote:
>> "Please try to change FreeTDS's client char set setting from UTF-8 to
>> EUCJP-MS as below.
>>
>> 1) add code EUCJP-MS to 'src/tds/encodings.h'.
>>
>> 2) change strings buffer from 256 to 100000 in 'src/tds/write.c'.
>>
>> 3) modify freetds.conf as such:  client charset=EUCJP-MS
>>
>>
>> I implemented 2) and 3) but for 1) did not know where to put the code
> in
>> the file.  Also, just wanted to know if you approve of this approach,
> in
>> general.  If so.....after these modifications, do I need to redo the
>> freetds "make" and "make install"?
> 
> Hi Laura, 
> 
> If these changes are necessary, yours would be the first case, and I'd
> like to understand why.  Adding code to encodings.h isn't baby simple:
> the
> file is generated because there are interlocking parts.  I suspect it
> would be better to add the name -- assuming that's required at all -- to
> alternative_character_sets.h and regenerate encodings.h.  And, yes, it
> would mean rebuilding and re-installing FreeTDS.  
> 
> But how do we know the solution lies in that direction?  To answer that,
> we'd have to know more about your environment.  To wit:
> 
> 1.  OS and version
> 2.  iconv library implementation
> 3.  encoding of data coming from the Oracle server
> 4.  encoding of single-byte data on the Microsoft server
> 5.  the gadget that's using FreeTDS (maybe)
> 
> I would also pursue getting a TDSDUMP log.  
> 
> I don't see how #2 will change anything.  Whether the buffer is 10 bytes
> or 10 million, the data are all converted and written.  
> 
> #1 and #3 are related.  They're based on the premise that FreeTDS is not
> #recognizing the encoding being used by the "client" i.e. the Oracle
> #server.  That could be true but, as I said, I'm skeptical.  
> 
> Can you log onto the gateway machine and use tsql to insert Japanese
> characters correctly?  That would demonstrate none of the above changes
> are needed.  
> 
> Regards, 
> 
> --jkl
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds
> 
> 
> This email is for the use of the intended recipient(s) only.  If you have 
> received this email in error, please notify the sender immediately and then 
> delete it.  If you are not the intended recipient, you must not keep, use, 
> disclose, copy or distribute this email without the author's prior 
> permission.  We have taken precautions to minimize the risk of transmitting 
> software viruses, but we advise you to carry out your own virus checks on 
> any attachment to this message.  We cannot accept liability for any loss 
> or damage caused by software viruses.  The information contained in this 
> communication may be confidential and may be subject to the attorney-client 
> privilege. If you are the intended recipient and you do not wish to receive 
> similar electronic messages from us in the future then please respond to the 
> sender to this effect.
> 
> _______________________________________________
> FreeTDS mailing list
> [email protected]
> http://lists.ibiblio.org/mailman/listinfo/freetds
>