Re: Unicode and Sybase univarchar

On Thu, 3 Jun 2010, Michael Peppler wrote:

> I just gave this a try - I'm under linux, with ASE 15.5. I created a table with a univarchar column, entered some data via isql, then wrote a minimal perl script to fetch the data.
>
> If I use a UTF8 locale (i.e. LANG=en_us.UTF8) I get the correct output.
> If I don't I do not get the correct output, at least for rows where non-ascii data has been entered into the table.

Define correct output here. From looking at the DBD::Sybase code, I don't 
see how it could possibly be right, because there's nothing in there to 
set the utf8 flag on the Perl string when the data is retrieved.

So even if I can work around the bizarro "bytes as a string" issue, I 
still won't have utf8 after the round trip.

My test script did something like this:

     my $dbh = ...; # set charset to utf8

     round_trip("em dash: \x{2014}");

     sub round_trip {
         my $unicode = shift;

         $dbh->do('DELETE FROM unicode');

         check($unicode);

         $dbh->do( 'INSERT INTO unicode (utest) VALUES (?)', {}, $unicode );

         my $rows = $dbh->selectall_arrayref('SELECT * FROM unicode');

         my $fromdb = $rows->[0][0];

         check($fromdb);

         my $chars = do {
             use bytes;

             join q{}, map { chr( eval '0x' . $_ ) } $fromdb =~ /(....)/g;
         };

         check($chars);

         my $decoded = decode( 'utf-8', $chars );

         check($decoded);
     }

     sub check {
         my $string = shift;

         print "$string is utf8? ",
             ( Encode::is_utf8($string) ? 'yes' : 'no' ),
             "\n";
     }

     __END__

     CREATE TABLE unicode (
       utest  univarchar(250)  NOT NULL,
     );


-dave

/*============================================================
http://VegGuide.org               http://blog.urth.org
Your guide to all that's veg      House Absolute(ly Pointless)
============================================================*/