Re: WSGI 2.0 Round 2: requirements and call for interest

This is a multi-part message in MIME format.
--===============1473493531805706170==
Content-Type: multipart/alternative;
 boundary="------------090301010601020508080405"

This is a multi-part message in MIME format.
--------------090301010601020508080405
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

Just to add my 2c - as another Django developer, I agree completely with=20
Aymeric here. My own experience was that the HTTP handling done by WSGI=20
(especially URL handing, HTTP header mangling, os.environ as a=20
destination - all due to CGI compatibility - and semi-broken unicode=20
handling) only made things harder for us. We would much rather have=20
dealt with raw streams of bytes and done all HTTP parsing ourselves.

Like Graham said, for HTTP/2 let's ignore the history of WSGI and start=20
from scratch with a API that actually serves us well.

Regards,

Luke


On 05/01/16 11:26, Cory Benfield wrote:
> Forwarding this message from the django-developers list.
>
> Hi Cory,
>
> I=E2=80=99m not subscribed to web-sig but I read the discussion there. =
Feel free to forward my answer to the group if you think it=E2=80=99s use=
ful.
>
> I have roughly the same convictions as Graham Dumpleton. If you want to=
 support HTTP/2 and WebSockets, don=E2=80=99t start with design decisions=
 anchored in CGI. Figure out what a simple and flexible API for these new=
 protocols would be, specify it, implement it, and make sure it degrades =
gracefully to HTTP/1. You may be able to channel most of the communicatio=
n through a single generator, but it=E2=80=99s unclear to me that this wi=
ll be the most convenient design.
>
> If you want to improve WSGI, here=E2=80=99s a list of mistakes or short=
comings in PEP 3333 that you can take a stab at. There=E2=80=99s a genera=
l theme: for a specification that looks at the future, I believe that mak=
ing modern PaaS-based deployments secure by default matters more than not=
 implementing anything beyond what=E2=80=99s available in legacy CGI-base=
d deployments.
>
> 1. WSGI is prone to header injection vulnerabilities issues by design d=
ue to the conversion of HTTP headers to CGI-style environment variables: =
if the server doesn=E2=80=99t specifically prevent it, X-Foo and X_Foo bo=
th become HTTP_X_Foo. I don=E2=80=99t believe it=E2=80=99s a good choice =
to destructively encode headers, expect applications to undo the damage s=
omehow, and introduce security vulnerabilities in the process. If mimicki=
ng CGI is still considered a must-have =E2=80=94 1% of current Python web=
 programmers may have heard about it, most of them from PEP 3333 =E2=80=94=
 then that burden should be pushed onto the server, not the application.
>
> 2. More generally, I fail to see how mixing HTTP headers, server-relate=
d inputs, and environment variables in a dict adds values. It prevents it=
erating on each collection separately. It only makes sense if not offerin=
g more features than CGI is a design goal; in that case, this discussion =
doesn=E2=80=99t serve a purpose anyway. It would be nicer and possibly mo=
re secure if the application received separately:
>
> a. Configuration information, which servers could read from environment=
 variables by default for backwards compatibility, but could also get thr=
ough more secure channels and restrict to what the application needs in o=
rder to better isolate it from the entire OS.
> b. Server APIs mandated by the spec, per request.
> c. HTTP headers, per request.
>
> 3. Stop pretending that HTTP is a unicode protocol, or at least stop ig=
noring reality when doing so. WSGI enforces ISO-8859-1-decoded str object=
s in the environ, which is just wrong. It=E2=80=99s all the more a surpri=
sing choice since this change was driven by Python 3, that UTF-8 is the c=
orrect choice, and that Python 3 defaults to UTF-8. Django has to re-enco=
de and re-decode before doing anything with HTTP headers: https://github.=
com/django/django/blob/d5b90c8e120687863c1d41cf92a4cdb11413ad7f/django/co=
re/handlers/wsgi.py#L231-L253
>
> 4. Normalize the way to tell the application about the original protoco=
l, IP address and port. When dev and ops responsibilities are separate, t=
his is clearly an ops responsibility, but due to the lack of standardizat=
ion devs end up dealing with this problem in custom middleware, when they=
 do it at all. Everyone keeps getting it wrong, which introduces security=
 vulnerabilities. Also it always breaks silently on infrastructure change=
s.
>
> 5. Improve request / response length handling and connection closure. A=
rmin and Graham have talked about in the past and know the topic better t=
han I do. There=E2=80=99s also a rejected PEP by Armin which made sense t=
o me.
>
> As you can see from these comments, I don=E2=80=99t quite share the des=
ign choices that led to WSGI as it currently stands. I think it will be e=
asier to build a new standard than evolve the current one.
>
> I hope this helps!
>
> Aymeric
>
>
> _______________________________________________
> Web-SIG mailing list
> [email protected]
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: https://mail.python.org/mailman/options/web-sig/l.plant.98=
%40cantab.net

--=20
"Trouble: Luck can't last a lifetime, unless you die young."
(despair.com)

Luke Plant || http://lukeplant.me.uk/


--------------090301010601020508080405
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html>
  <head>
    <meta content=3D"text/html; charset=3Dutf-8" http-equiv=3D"Content-Ty=
pe">
  </head>
  <body bgcolor=3D"#FFFFFF" text=3D"#000000">
    Just to add my 2c - as another Django developer, I agree completely
    with Aymeric here. My own experience was that the HTTP handling done
    by WSGI (especially URL handing, HTTP header mangling, os.environ as
    a destination - all due to CGI compatibility - and semi-broken
    unicode handling) only made things harder for us. We would much
    rather have dealt with raw streams of bytes and done all HTTP
    parsing ourselves.<br>
    <br>
    Like Graham said, for HTTP/2 let's ignore the history of WSGI and
    start from scratch with a API that actually serves us well.<br>
    <br>
    Regards,<br>
    <br>
    Luke<br>
    <br>
    <br>
    <div class=3D"moz-cite-prefix">On 05/01/16 11:26, Cory Benfield wrote=
:<br>
    </div>
    <blockquote
      cite=3D"mid:CA133565-1987-4711-8D89-CCC18DE900BE-4cWZ1Abt8O3QXOPxS62xeg@public.gmane.org"
      type=3D"cite">
      <pre wrap=3D"">Forwarding this message from the django-developers l=
ist.

Hi Cory,

I=E2=80=99m not subscribed to web-sig but I read the discussion there. Fe=
el free to forward my answer to the group if you think it=E2=80=99s usefu=
l.

I have roughly the same convictions as Graham Dumpleton. If you want to s=
upport HTTP/2 and WebSockets, don=E2=80=99t start with design decisions a=
nchored in CGI. Figure out what a simple and flexible API for these new p=
rotocols would be, specify it, implement it, and make sure it degrades gr=
acefully to HTTP/1. You may be able to channel most of the communication =
through a single generator, but it=E2=80=99s unclear to me that this will=
 be the most convenient design.

If you want to improve WSGI, here=E2=80=99s a list of mistakes or shortco=
mings in PEP 3333 that you can take a stab at. There=E2=80=99s a general =
theme: for a specification that looks at the future, I believe that makin=
g modern PaaS-based deployments secure by default matters more than not i=
mplementing anything beyond what=E2=80=99s available in legacy CGI-based =
deployments.

1. WSGI is prone to header injection vulnerabilities issues by design due=
 to the conversion of HTTP headers to CGI-style environment variables: if=
 the server doesn=E2=80=99t specifically prevent it, X-Foo and X_Foo both=
 become HTTP_X_Foo. I don=E2=80=99t believe it=E2=80=99s a good choice to=
 destructively encode headers, expect applications to undo the damage som=
ehow, and introduce security vulnerabilities in the process. If mimicking=
 CGI is still considered a must-have =E2=80=94 1% of current Python web p=
rogrammers may have heard about it, most of them from PEP 3333 =E2=80=94 =
then that burden should be pushed onto the server, not the application.

2. More generally, I fail to see how mixing HTTP headers, server-related =
inputs, and environment variables in a dict adds values. It prevents iter=
ating on each collection separately. It only makes sense if not offering =
more features than CGI is a design goal; in that case, this discussion do=
esn=E2=80=99t serve a purpose anyway. It would be nicer and possibly more=
 secure if the application received separately:

a. Configuration information, which servers could read from environment v=
ariables by default for backwards compatibility, but could also get throu=
gh more secure channels and restrict to what the application needs in ord=
er to better isolate it from the entire OS.
b. Server APIs mandated by the spec, per request.
c. HTTP headers, per request.

3. Stop pretending that HTTP is a unicode protocol, or at least stop igno=
ring reality when doing so. WSGI enforces ISO-8859-1-decoded str objects =
in the environ, which is just wrong. It=E2=80=99s all the more a surprisi=
ng choice since this change was driven by Python 3, that UTF-8 is the cor=
rect choice, and that Python 3 defaults to UTF-8. Django has to re-encode=
 and re-decode before doing anything with HTTP headers: <a class=3D"moz-t=
xt-link-freetext" href=3D"https://github.com/django/django/blob/d5b90c8e1=
20687863c1d41cf92a4cdb11413ad7f/django/core/handlers/wsgi.py#L231-L253">h=
ttps://github.com/django/django/blob/d5b90c8e120687863c1d41cf92a4cdb11413=
ad7f/django/core/handlers/wsgi.py#L231-L253</a>

4. Normalize the way to tell the application about the original protocol,=
 IP address and port. When dev and ops responsibilities are separate, thi=
s is clearly an ops responsibility, but due to the lack of standardizatio=
n devs end up dealing with this problem in custom middleware, when they d=
o it at all. Everyone keeps getting it wrong, which introduces security v=
ulnerabilities. Also it always breaks silently on infrastructure changes.

5. Improve request / response length handling and connection closure. Arm=
in and Graham have talked about in the past and know the topic better tha=
n I do. There=E2=80=99s also a rejected PEP by Armin which made sense to =
me.

As you can see from these comments, I don=E2=80=99t quite share the desig=
n choices that led to WSGI as it currently stands. I think it will be eas=
ier to build a new standard than evolve the current one.

I hope this helps!

Aymeric
</pre>
      <br>
      <fieldset class=3D"mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap=3D"">_______________________________________________
Web-SIG mailing list
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:[email protected]">=
[email protected]</a>
Web SIG: <a class=3D"moz-txt-link-freetext" href=3D"http://www.python.org=
/sigs/web-sig">http://www.python.org/sigs/web-sig</a>
Unsubscribe: <a class=3D"moz-txt-link-freetext" href=3D"https://mail.pyth=
on.org/mailman/options/web-sig/l.plant.98%40cantab.net">https://mail.pyth=
on.org/mailman/options/web-sig/l.plant.98%40cantab.net</a>
</pre>
    </blockquote>
    <br>
    <pre class=3D"moz-signature" cols=3D"72">--=20
"Trouble: Luck can't last a lifetime, unless you die young."=20
(despair.com)

Luke Plant || <a class=3D"moz-txt-link-freetext" href=3D"http://lukeplant=
.me.uk/">http://lukeplant.me.uk/</a>
</pre>
  </body>
</html>

--------------090301010601020508080405--

--===============1473493531805706170==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Web-SIG mailing list
[email protected]
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org

--===============1473493531805706170==--