Re: utf8 horror stories

On Fri, 17 Sep 2004, Autrijus Tang wrote:
> > > On Thu, Sep 16, 2004 at 12:52:19PM -0600, John Williams wrote:
> > > > Perl desperately needs a "default-to-utf8" mode.
> > >
> > > It's called "use encoding 'utf8';".  It's also subtly enabling
> > > lots of other knobs.  See encoding::warnings on CPAN for details.
> >
> > Have you actually tried it?
>
> Why yes, I'm one of its authors. :)

Oh good, then I can complain to you a bit.  :)

> > That only sets your source file and STDIN/STDOUT to utf8.
>
> Yes, but it enables utf8 semantics globally instead of the default
> latin1 promotion, so high-bit characters you read from other handles
> will be treated correctly when joined with utf8 data.

OK, I see where it says that in the docs now.  But it doesn't seem to be
working for me.

Right now I'm doing this (in my local copy of Mason::Buffer::receive),
because it doesn't matter whether utf8 is on or off once it gets to
Apache:

        use Encode;
        foreach ( grep { defined } @_) {
                Encode::_utf8_off($_);
                ${ $self->{buffer} } .= $_;
        }

I tried changing it to this:

        use encoding 'utf8';
        ${ $self->{buffer} } .= $_ foreach grep { defined } @_;

But all the doubly-encoded utf8 strings came back, indicating that it is
still doing the latin1->utf8 conversion on the strings which are "off".

I'm using perl-5.8.3, encoding-1.48, mod_perl-1.99_12.  Do I need to
upgrade something?

~ John Williams




-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM.
Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php