Add a high-level HTTP client API based on the HTTP client implemented for #886

New submission from exarkun <[email protected]>:

#886 adds a new HTTP client implementation.  The API provided by that code is very low-level though.  It is essentially a protocol with a request method.  Connection setup, URL interpretation, redirects, caching, cookies, proxies, authentication, etc are all left up to someone else. There should be a higher-level API which allows for all of these things.  The actual implementation of each of these higher-level features may be left up to some other ticket, and other similar features may be left out of Twisted entirely - extension by third parties should be easy and straightforward with this new API.

Here is my proposed API.  There will be a single primary method, `request`.  It will accept 3 or 4 arguments: the request method, URI, headers, and body.  The method and URI will be strings (later we may want to use `URLPath` or something for the URI, but I don't think it's ready now); the headers will be an instance of `twisted.web.http_headers.Headers`; and the body will be an optional `IEntityBodyProducer` provider.  The return value will be like `HTTP11ClientProtocol.request`.

This ticket will be resolved by the simplest possible implementation of this API: a class (with a parameterized reactor) which sets up a new connection for each request and mostly just passes through to `HTTP11ClientProtocol` (but it will insert ''required'' headers).

More advanced features will generally be implemented by layering other things on top of this.  For example, there should eventually be classes such as `CachingClient`, `ProxyEnabledClient`, and `CookieEnabledClient` which allow an application to construct an object like this:

{{{
#!python
client = CachingClient(CookieEnabledClient(ProxyEnabledClient(BasicClient(reactor))))
}}}

And then use it just as it would have used `BasicClient`:

{{{
#!python
responseDeferred = client.request('GET', 'http://example.com/foo', Headers({'Accept': ['en']}))
}}}


----------
Type     : enhancement
Component: web
Keywords : httpclient
Priority : normal
Nosy     : 
----------
http://twistedmatrix.com/trac/ticket/3987