twisted/web/http.py: HttpClient violates RFC 2616 when port != 80

New submission from Vultaire <None>:

This one bit me when I relocated a WordPress blog from port 80 to port 8080 on a server, and was trying to use getPage to grab a page from the blog.  (getPage indirectly calls HttpClient.)

The HttpClient protocol does not seem to properly specify the "host" header in HTTP requests, and because of doing so, is ending up in an infinite redirect loop.

Specifically, given the url "http://domu-XX-XX-XX-XX-XX-XX.compute-1.internal:8080/wordpress/", here's what happens:
{{{
sendCommand: GET /wordpress/ HTTP/1.0
sendHeader: Host: domu-XX-XX-XX-XX-XX-XX.compute-1.internal
sendHeader: User-Agent: Twisted PageGetter
endHeaders

lineReceived: HTTP/1.0 301 Moved Permanently
lineReceived: Date: Thu, 28 May 2009 04:47:19 GMT
lineReceived: Server: Apache/2.2.9 (Ubuntu) PHP/5.2.6-2ubuntu4.2 with Suhosin-Patch
lineReceived: X-Powered-By: PHP/5.2.6-2ubuntu4.2
lineReceived: X-Pingback: http://domU-XX-XX-XX-XX-XX-XX.compute-1.internal:8080/wordpress/xmlrpc.php
lineReceived: Location: http://domu-XX-XX-XX-XX-XX-XX.compute-1.internal:8080/wordpress/
lineReceived: Vary: Accept-Encoding
lineReceived: Content-Length: 0
lineReceived: Connection: close
lineReceived: Content-Type: text/html; charset=UTF-8
lineReceived: 

[...loops, raises exception, exits.]
}}}

Connecting via telnet to port 8080 and doing the same commands by hand generates the same response, of course.

However, if I tack ":8080" to the host header:

{{{
sendCommand: GET /wordpress/ HTTP/1.0
sendHeader: Host: domu-XX-XX-XX-XX-XX-XX.compute-1.internal:8080
sendHeader: User-Agent: Twisted PageGetter
endHeaders

lineReceived: HTTP/1.0 200 OK
lineReceived: Date: Thu, 28 May 2009 05:08:32 GMT
lineReceived: Server: Apache/2.2.9 (Ubuntu) PHP/5.2.6-2ubuntu4.2 with Suhosin-Patch
lineReceived: X-Powered-By: PHP/5.2.6-2ubuntu4.2
lineReceived: X-Pingback: http://domU-XX-XX-XX-XX-XX-XX.compute-1.internal:8080/wordpress/xmlrpc.php
lineReceived: Vary: Accept-Encoding
lineReceived: Connection: close
lineReceived: Content-Type: text/html; charset=UTF-8
lineReceived:

[output follows]
}}}

[http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23 RFC 2616] does specify that the port should be specified on this header, else port 80 will be implied.

I know not all cases are affected; Apache seems to be pretty forgiving and I can retrieve static pages fine.  But when I go for anything in WordPress, then I run into problems.


----------
Type     : defect
Component: core
Keywords : 
Priority : normal
Nosy     : 
----------
http://twistedmatrix.com/trac/ticket/3857