Re: Silent removed parameters when posting UTF8

On Fri, Sep 19, 2008 at 02:14:25PM +0200, Gisle Aas wrote:
> I wonder if this is a bug in perl itself.  With perl-5.8.8 I get:
> 
>     count=123&works=%E2%98%BA&does_not_work=&foo=bar
> 
> as you showed, but with perl-5.8.9(tobe) and perl-5.10.0 I get:
> 
>     count=123&works=%C3%A2%C2%98%C2%BA&does_not_work=%E2%98%BA&foo=bar
> 
> so now it's the 'works' case that does_not_work but in a different way :(

That's bothersome.

Would you agree that the url encoding must be done on octets?  If so,
then I think it would be correct to issue a warning if the utf8 flag
is on.

I think doing it in query_form would be the best place as it tells the
user what specific item had the utf8 flag on.


        while (my($key,$vals) = splice(@new, 0, 2)) {
            $key = '' unless defined $key;

            warn "key [$key] has utf8 flag set and may not url-encode"
                if utf8::is_utf8( $key );

	    $key =~ s/([;\/?:@&=+,\$\[\]%])/$URI::Escape::escapes{$1}/g;
	    $key =~ s/ /+/g;
	    $vals = [ref($vals) eq "ARRAY" ? @$vals : $vals];
            for my $val (@$vals) {
                $val = '' unless defined $val;

                warn "value [$val] for key [$key] has utf8 flag set and may not url-encode"
                    if utf8::is_utf8( $val );

		$val =~ s/([;\/?:@&=+,\$\[\]%])/$URI::Escape::escapes{$1}/g;
  

Actually, I guess I'd argue to die, but that might break existing
code.  After all, if trying to url-encode characters might result in
dropped or altered data that's pretty serious.

Thanks,

-- 
Bill Moseley
[email protected]
Sent from my iMutt