Collating follow-up on the future of WSGI

--===============4672455146903303974==
Content-Type: multipart/signed; boundary="Apple-Mail=_8E0261DE-C700-40E9-A9A9-3193841B7D10"; protocol="application/pgp-signature"; micalg=pgp-sha256


--Apple-Mail=_8E0261DE-C700-40E9-A9A9-3193841B7D10
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

All,

Thanks so much for your feedback to my original request for comments on =
the future of WSGI. You provided a ton of really useful feedback: when =
printed out on my printer it ended up at about 50 pages of information =
that was really engaging reading. I also want to thank you all for =
keeping the tone of the discussion so positive. On an topic like this =
one it can get tricky and emotionally charged very easily, and you did a =
great job of avoiding that problem.

I spent a few hours this morning going over your feedback and trying to =
extract some common threads. Altogether I believe that most participants =
were mostly in agreement over the direction we should take, with a few =
outliers in each case. I=E2=80=99d like to summarise what I believe were =
the big take-aways from that discussion to confirm that I=E2=80=99ve =
understood everyone.

I=E2=80=99d also like the members of this SIG to take this opportunity =
to discuss these proposals more concretely. Rather than expressing our =
sentiments about WSGI and its future more generally, I want people to =
critique and offer opinions on *these specific proposals*. The goal here =
is to get an understanding of whether these are worth doing, how we need =
to prioritise the work, and whether there=E2=80=99s anything we=E2=80=99re=
 missing.

The below is formatted in restructured text for clarity, because it=E2=80=99=
s quite a lot of information.


Concrete Proposals
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

WSGI
----

Overall, there was strong and fairly unanimous sentiment that WSGI =
itself should more or less be left alone. Minor adjustments would be =
valuable, and we should pursue them, but the contributors do not believe =
that making a substantial revision to WSGI would be adviseable.

I therefore propose we revise the WSGI specification to WSGI 1.1 and =
then consider it "final". The following points were raised for revising =
WSGI.

Asynchronous WSGI
~~~~~~~~~~~~~~~~~

This was generally regarded as too substantial a change to shoehorn into =
WSGI.

Benoit proposed that we could achieve this change by adding a correlator =
to WSGI. This would allow servers to associate a given call to =
``write()`` with a specific request/response (most useful in HTTP/2 =
where this could correspond to a stream ID). This would then allow WSGI =
to transition to a purely callback-based model that could in principle =
cohabitate with an async protocol in Python.

This proposal is worth highlighting not becuase I think we should pursue =
it with our revision of WSGI, but because it's worth considering for any =
future specification we come up with. Note also that Graham pointed out =
that this would require some careful rethinking of reads from =
``wsgi.input``.

Server Push
~~~~~~~~~~~

We can support HTTP/2 server push using Link headers. This could =
optionally be supplemented by defining a WSGI extension that provides a =
callable for doing server push, which would be a mild improvement over =
the Link header approach. A simple proposal for how to do this can =
easily be implemented without revising the WSGI specification, and may =
not even need to be enshrined in a PEP. However, if we revise the WSGI =
PEP we may want to provide a small section that indicates how to add =
these headers.

Socket Escape Hatch
~~~~~~~~~~~~~~~~~~~

Aside from Benoit, server operators were unanimously dismissive of the =
idea of a socket 'escape hatch'. In general it seems like servers would =
not be capable of achieving this. I think, therefore, this idea is =
unworkable.

Chunked Transfer Encoding
~~~~~~~~~~~~~~~~~~~~~~~~~

It would be nice to formalise chunked transfer encoding in WSGI. =
Currently there is no way to signal to applications that chunked =
transfer encoding is in use by the client, or for the application to =
request it from the server.

This seemed to be a low priority work item, but if we can make this =
enhancement easily then it's worth considering.

Bytes and Unicode
~~~~~~~~~~~~~~~~~

Several contributors expressed dissatisfaction for PEP-3333's approach =
to headers (namely, Latin-1-encoded Unicode strings), and expressed a =
preference for using bytestrings. If we attempt this change, we have a =
backward-compatibility concern, so we may have to live with this =
decision.

Regardless, this should be taken as a warning sign for any other =
specification we attempt: more on that later.

REQUEST_URI environ variable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Multiple contributors expressed an interest in bringing this environment =
variable into WSGI directly, making it a required part of the environ =
dictionary. An alternative name for this was RAW_URI.

Header Joining and Name Normalization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Armin pointed out that PEP-3333 does not mention what should happen to =
normalize header names and to join header fields that appear multiple =
times in a header block. CGI does not appear to proscribe a behaviour =
here either.

A revision of PEP-3333 should cover how header names get normalised =
(where the answer is basically "like CGI"), and how servers should join =
headers together (and logically, how applications can split them). =
Mostly this is simply codifying existing practice.

Content Lengths
~~~~~~~~~~~~~~~

We should clarify in the new specification that an application that =
reads beyond the logical length of the request as given by =
CONTENT_LENGTH will have its reads return immediately with the empty =
string. Servers are required to police that logic. This is codifying =
existing practice, and would also make CONTENT_LENGTH purely advisory.

File Wrapper
~~~~~~~~~~~~

Graham identified two specific problems with ``wsgi.file_wrapper``: =
specifically, the bad example (which doesn't close the file descriptor =
properly), and the larger problem about the overly general specification =
of the ``file_wrapper`` as he's blogged about in the past.

In general, we may want to identify ``file_wrapper`` as a point of =
contention, and consider more fundamental revisions to this under-used =
part of the WSGI specification. We may also want to consider simply =
dropping it entirely, or pulling it out to an extension specification =
rather than leaving it in WSGI 1.1.

Websockets and HTTP/2
---------------------

Here, the consensus seemed to be that we should investigate whether we =
can come up with WSGI-equivalent APIs for Websockets and HTTP/2. The =
goal here would be to allow servers that are interested in offering =
richer APIs than WSGI can provide would be able to do so.

This is particularly valuable for HTTP/2, which can function totally =
fine when used with WSGI but which can have much richer functionality =
made available to the application via a different API if that =
application requires it.

As a SIG we should begin to consider what these APIs should look like. =
We should also bear in mind the more advanced asynchronous interfaces =
(see below), and ensure that we can easily proxy these APIs to the more =
advanced ones, as we can with WSGI to ASGI.

Meta API
~~~~~~~~

Graham suggested, along with these more advanced APIs, that we may want =
a meta-API or API discovery API that would allow servers to target a =
single entry point which then provides access to the callables for WSGI, =
the Websockets API, and the HTTP/2 API. We should bear this idea in =
mind, but I don't believe we should worry too much about whether we =
actually build it or not at this time: it can definitely follow the =
prototyping work on the HTTP/2 specific APIs.

Asynchronous Python
-------------------

There was an overall positive reaction to introducing *something* that =
fills the WSGI role for applications that use asynchronous Python =
frameworks.

Exactly how this would look is unclear at this stage, and this should be =
thought of as a long-term goal for this SIG. Amber Brown is clearly =
working on an idea, and we also have Andrew's work on ASGI, either of =
which could be used as a basis for a future PEP.

There was, however, a clear consensus that this should be a clean break =
from WSGI: while a server implementing this specification should be able =
to call into WSGI applications with a small shim, there is no =
requirement that an *application* using this specification should be =
callable from a WSGI server.

We should absolutely bear this in mind, and we should encourage =
specification work to be done here when things are ready, but this is =
definitely not going to happen as quickly as the other ideas above.



Please let me know what you think of this set of proposals. If =
everything is in good shape, I=E2=80=99ll begin working on some of these =
items, hopefully with the help of some of the other members of this SIG!

Cory

--Apple-Mail=_8E0261DE-C700-40E9-A9A9-3193841B7D10
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCAAGBQJWnmpzAAoJEHD+F/imQ+FbinwP/0eM2AD3QUtj9DmCRcM0a+S7
NNluUcfT3zDCiz90yPknmU2VIb0BVDHmM25d8fsP5+emqTDL7BjATiIN+ssNUi3b
RYK+dGLHBEWOfpO6MdHmUqmKSfB5jKgM1OmLKee15j8rj5R0qJHSJZGEwWQvXtq+
RqfUBVnLlQqHfsR+1RG0YE3DpsnDSos/4/wdHdAAfP+5aoPLud5ceFcAHc2N52+q
9kltFw98yYe1CNdDTqcwqVA6K3HQRSLnFOzuafjjvrGPrsPn40cXZb3DhF6rXzdu
5qDcXDA83VT2t7OAEm3SoIgC5jPN3RiwgGOToBIoUw/6qJyLBaxBJI5TcsM46yGq
/8P3Fr3YgQZyIOwbnWrinINiCnS6RigcnLa1XVwkWjahZ7d1Ab15EhSRs/8JmMxU
A4jvSmgBivgulN267JVEzlUthrYyJ2spxUEA2bYIZvkkdnyYNuY4vvSPGqYZ8GAr
UT63+cC1sNHr+CqitoU6fGzWY6B6q+WaZkmLmI7mPBp300qUR7e0rYgh/bLUWwiN
T6kRz9DDQJnHjiO+F8ci6wUFvyovkSi2olGGAmda4NPuevWdavLkHu2NNMB4DlBQ
3qiV6cdLtSUfVrVn35PtGevR8tzBz189WNYGmQZNFNBjhOGj0/kgWFtsI1UEs2jV
SKyMJ/W+AmCkwNtv7/0E
=sNNx
-----END PGP SIGNATURE-----

--Apple-Mail=_8E0261DE-C700-40E9-A9A9-3193841B7D10--

--===============4672455146903303974==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Web-SIG mailing list
[email protected]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org

--===============4672455146903303974==--