Re: URL parameter encoding bug

Mark,

First of all, ensure that you specify -encoding to javac for your  
source file; if you don't, javac uses the local default encoding to  
interpret your source code, which is almost surely not what you want.  
You can also stick to ASCII and use a \u00A3 to specify £.


On May 17, 2006, at 6:06 AM, Mark Hobson wrote:

> I believe I've encountered a parameter encoding bug with URL  
> encoded post requests in httpunit.  See the below testcase for an  
> example - it passes on windows, but fails on linux.
>
> I believe this is due to the call to String.getBytes() in  
> URLEncodedMessageBody.writeTo() which uses the platforms native  
> encoding - i.e. ISO-8859-1 in windows, which is the correct  
> encoding for HTTP, and then UTF-8 in linux, which is an incorrect  
> encoding for HTTP.

That's not at all correct. Although it defines a default of  
ISO-8859-1, HTTP does not proscribe a “correct” character set for any  
purpose[1]. In lieu of an explicit request encoding, RFC 3986 could  
be interpreted to be dominant for form bodies—and it proscribes UTF-8  
(not ISO-8859-1) for URL encoding[3]! In practice, however, the local  
default encoding at the server will prevail in the absence of a  
specified encoding, so the only safe behavior is for a client is to  
specify charset in its Content-Type.

The charset which prevails in this decode operation should be the one  
specified in the Content-Type header. In HTML, this is set using form/ 
@accept-charset, which defaults to the document charset[2].

The bug, if any, is that the generated request does not include the  
character set…

     "Content-Type: application/x-www-form-encoded; charset="
       + Charset.defaultCharset.name()

Or that ServletUnit (in your test case) does not respect it when  
decoding the request body.

— G

[1] http://www.rfc.net/rfc2616.html#s3.4
[2] http://www.w3.org/TR/html4/interact/forms.html#h-17.3
[3] http://www.rfc.net/rfc3986.html