Re: unicode

Murray, Laura (US) wrote:
> Japanese
> and Chinese characters appear properly in our Oracle database, but
> become garbled when we try to send them to the SQL Server database using
> freetds.  Does freetds support Unicode?  

http://msdn.microsoft.com/en-us/library/aa214408(SQL.80).aspx

It depends what "support" and "Unicode" means.  You don't say what
encoding your Oracle gateway is using to represent Japanese.  UCS-2? 
UTF-8?  Big5?  How about on the server?  What is the datatype and
collation of the target column?  

You also don't say what technology you're using.  I assume ODBC, but I
don't know how ODBC is being used to move the data.  Is the gateway
constructing SQL statements?  Is it using prepared statements with
parameters?  

It also depends what you mean by "FreeTDS"; you don't mention a version. 

A TDSDUMP log would answer these questions.  

If I were to venture a guess, it would be that your configuration is
relying on defaults, and FreeTDS understands the client encoding to be ISO
8859-1.  The gateway meanwhile is using parameterized queries and filling
the parameter buffers using another encoding, perhaps utf-8.  What lands
on the other side -- what's stored in SQL Server -- isn't right.  

I doubt the gateway is using UCS-2 because then your English data would
also be garbled.  

Here's a test you could do without logging:

1.  Insert a single Chinese character into your Oracle table. Be sure you
know the Unicode codepoint. 
2.  See what binary value was stored on the Microsoft server with e.g.:

	select cast(colname as binary) from tablename

3.  Repeat with columns of type VARCHAR, NVARCHAR, and IMAGE. 

Based on my above guesses, I would expect the stored values to be the
result of treating the Unicode input as ISO 8859-1 text and converting it
to the server-declared encoding for the column.  

HTH.  

--jkl