Re: URL parameter encoding bug

Hi Gordon,

Thanks for the reply.

On 17/05/06, Gordon Henriksen <[email protected]> wrote:
> First of all, ensure that you specify -encoding to javac for your source
> file; if you don't, javac uses the local default encoding to interpret your
> source code, which is almost surely not what you want. You can also stick to
> ASCII and use a \u00A3 to specify £.

Sure, I'm using eclipse with workspace text file encoding set to
UTF-8, so I should be okay there.

> That's not at all correct. Although it defines a default of ISO-8859-1, HTTP
> does not proscribe a "correct" character set for any purpose[1]. In lieu of
> an explicit request encoding, RFC 3986 could be interpreted to be dominant
> for form bodies—and it proscribes UTF-8 (not ISO-8859-1) for URL
> encoding[3]! In practice, however, the local default encoding at the server
> will prevail in the absence of a specified encoding, so the only safe
> behavior is for a client is to specify charset in its Content-Type.
>
> The charset which prevails in this decode operation should be the one
> specified in the Content-Type header. In HTML, this is set using
> form/@accept-charset, which defaults to the document charset[2].

Great, thanks for the clarification.  I wasn't too sure about this but
figured someone would set me straight ;)

> The bug, if any, is that the generated request does not include the
> character set…
>
>     "Content-Type: application/x-www-form-encoded; charset="
>       + Charset.defaultCharset.name()

I've just added the following to the start of the test case to ensure this:

    HttpUnitOptions.setPostIncludesCharset(true);

And looking at the source this should now be included.  As before,
this passes on windows but fails on linux.

> Or that ServletUnit (in your test case) does not respect it when decoding
> the request body.

Which is what I believe is happening.  The charset is being added to
the request, but the String.getBytes() in
URLEncodedMessageBody.writeTo() is ignoring it by using the platforms
native encoding.

Interestingly, if I add
HttpUnitOptions.setDefaultCharacterSet("UTF-8") to the test case then
it works on both windows and linux.  Can anyone draw any conclusions
from this?

Cheers for the help,

Mark


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0709&bid&3057&dat1642