Re: Inviting feedback on my proposed "ASGI" spec
Andrew Godwin <[email protected]> Fri, 11 Mar 2016 10:05:07 -0800
| Newsgroups | gmane.comp.python.web |
|---|---|
| Message-ID | <CAFwN1uprrFRrs7MT034Ln0awyxJU5xZkfvdZ_cpgzPUVGgvsFA@mail.gmail.com> |
--===============4265750724743913954== Content-Type: multipart/alternative; boundary=001a113cf5ac9b5c70052dc9c6c5 --001a113cf5ac9b5c70052dc9c6c5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Yes, I thought that was the case. I think adding lowercase normalisation to header names to the spec would be sensible (daphne already does this, but I'd like to make it reliable upon) Andrew On Fri, Mar 11, 2016 at 10:03 AM, Collin Anderson <[email protected]> wrote: > http2 makes all header names lowercase > > On Fri, Mar 11, 2016 at 12:59 PM, Andrew Godwin <[email protected]> > wrote: > >> One thing I did want to ask - is it worth still squashing everything dow= n >> to the same case? Daphne already clears out headers with _ in them to av= oid >> that CVE about it, and header case is never semantic, or so I thought? >> >> Andrew >> >> On Fri, Mar 11, 2016 at 9:56 AM, Andrew Godwin <[email protected]> >> wrote: >> >>> >>> >>> On Fri, Mar 11, 2016 at 2:28 AM, Cory Benfield <[email protected]> >>> wrote: >>> >>>> >>>> On 10 Mar 2016, at 23:56, Andrew Godwin <[email protected]> wrote: >>>> >>>> I would indeed want to require servers to always fold headers together >>>> into a comma-separated list, as that's what the RFC says, and it then = means >>>> applications only have to deal with one kind of multi-header! >>>> >>>> >>>> Wellllll=E2=80=A6.kinda? >>>> >>>> The RFC says that multiple headers are *semantically equivalent* to th= e >>>> joined form, but does not in any sense require that it be done. (The >>>> normative language in RFC 7230 is MAY.) >>>> >>>> I had this discussion recently with Brian Smith: while there is only >>>> one correct way to fold/unfold headers, anywhere on the spectrum betwe= en >>>> completely folded and completely unfolded is a perfectly valid >>>> representation of the HTTP header block. This means that there=E2=80= =99s no *rules* >>>> about how a server is supposed to do it, at least from the IETF. ASGI = is of >>>> course totally allowed to add its own rules, and requiring that they b= e >>>> folded is not terrible. >>>> >>>> FWIW, in my experience, I=E2=80=99ve found that =E2=80=9Clist of tuple= s=E2=80=9D is really the >>>> most likely to be correct way to represent a header block, because it >>>> provides some assurances to the user that the header block has not bee= n >>>> aggressively transformed from how it was sent on the wire. While the >>>> *rules* are that the folded representation is supposed to be semantica= lly >>>> equivalent to the unfolded representation, there is nonetheless some >>>> information implicit in those headers being separate. >>>> >>>> My intuition when writing this kind of thing is to pass applications >>>> (like Django) the most meaningful representation I can, and then allow= the >>>> application to make its own decisions about what meaning they=E2=80=99= re willing to >>>> lose. That=E2=80=99s why I=E2=80=99d advocate for =E2=80=9Clist of two= -tuples of bytestrings=E2=80=9D as >>>> the representation. However, I don=E2=80=99t think there=E2=80=99s any= thing *wrong* with >>>> forcing the headers to be joined by the server where possible: it=E2= =80=99s just >>>> not how I=E2=80=99d do it. ;) >>>> >>>> Set-cookie is the annoying thing here, though. That's why it's dict >>>> inbound and list of tuples outbound right now, and I just don't know i= f I >>>> want to make the inbound one a list of tuples too, given I do definite= ly >>>> want to force servers to concat headers together (unless I find any >>>> examples of that screwing things up) >>>> >>>> >>>> You could make the inbound one a list of tuples but still require that >>>> the servers concat headers. The rule then would be that it needs to be >>>> possible for an application to say `dict(headers)` without any loss of >>>> meaning. >>>> >>> >>> Yes, I think this is a good argument - my worry has always been that th= e >>> "no multiples" is more of a soft rule that some clients might break or = some >>> apps might rely on the ordering/multiplicity of things, so preserving i= t is >>> _probably_ helpful (and as you say, it lets the header names go back to >>> bytestrings). >>> >>> I'll modify the spec and then update Daphne and Channels to match; I ca= n >>> leave Channels parsing both types for a bit, at least. >>> >>> Collin's point about http2's handling of headers is on point, too - if >>> the new spec is deliberately thinned down to that point but no further, >>> it's probably wise to follow them since they know much more about it th= an I >>> do. >>> >>> Andrew >>> >> >> >> _______________________________________________ >> Web-SIG mailing list >> [email protected] >> Web SIG: http://www.python.org/sigs/web-sig >> Unsubscribe: >> https://mail.python.org/mailman/options/web-sig/cmawebsite%40gmail.com >> >> > --001a113cf5ac9b5c70052dc9c6c5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Yes, I thought that was the case. I think adding lowercase= normalisation to header names to the spec would be sensible (daphne alread= y does this, but I'd like to make it reliable upon)<div><br></div><div>= Andrew</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote"= >On Fri, Mar 11, 2016 at 10:03 AM, Collin Anderson <span dir=3D"ltr"><<a= href=3D"mailto:[email protected]" target=3D"_blank">[email protected]= m</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi= n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">= http2 makes all header names lowercase</div><div class=3D"gmail_extra"><br>= <div class=3D"gmail_quote"><div><div class=3D"h5">On Fri, Mar 11, 2016 at 1= 2:59 PM, Andrew Godwin <span dir=3D"ltr"><<a href=3D"mailto:andrew@aerac= ode.org" target=3D"_blank">[email protected]</a>></span> wrote:<br></d= iv></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex"><div><div class=3D"h5"><div dir=3D"= ltr">One thing I did want to ask - is it worth still squashing everything d= own to the same case? Daphne already clears out headers with _ in them to a= void that CVE about it, and header case is never semantic, or so I thought?= <span><font color=3D"#888888"><div><br></div><div>Andrew</div></font></span= ></div><div><div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">= On Fri, Mar 11, 2016 at 9:56 AM, Andrew Godwin <span dir=3D"ltr"><<a hre= f=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&= gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 = 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><br><d= iv class=3D"gmail_extra"><br><div class=3D"gmail_quote"><span>On Fri, Mar 1= 1, 2016 at 2:28 AM, Cory Benfield <span dir=3D"ltr"><<a href=3D"mailto:c= [email protected]" target=3D"_blank">[email protected]</a>></span> wrote:= <br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef= t:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-word"><br>= <div><span><blockquote type=3D"cite"><div>On 10 Mar 2016, at 23:56, Andrew = Godwin <<a href=3D"mailto:[email protected]" target=3D"_blank">andrew@= aeracode.org</a>> wrote:</div><div><div dir=3D"ltr" style=3D"font-family= :Helvetica;font-size:12px;font-style:normal;font-weight:normal;letter-spaci= ng:normal;text-align:start;text-indent:0px;text-transform:none;white-space:= normal;word-spacing:0px"><div class=3D"gmail_extra"><div class=3D"gmail_quo= te"><div><br></div><div>I would indeed want to require servers to always fo= ld headers together into a comma-separated list, as that's what the RFC= says, and it then means applications only have to deal with one kind of mu= lti-header!</div></div></div></div></div></blockquote><div><br></div></span= ><div>Wellllll=E2=80=A6.kinda?</div><div><br></div><div>The RFC says that m= ultiple headers are *semantically equivalent* to the joined form, but does = not in any sense require that it be done. (The normative language in RFC 72= 30 is MAY.)</div><div><br></div><div>I had this discussion recently with Br= ian Smith: while there is only one correct way to fold/unfold headers, anyw= here on the spectrum between completely folded and completely unfolded is a= perfectly valid representation of the HTTP header block. This means that t= here=E2=80=99s no *rules* about how a server is supposed to do it, at least= from the IETF. ASGI is of course totally allowed to add its own rules, and= requiring that they be folded is not terrible.</div><div><br></div><div>FW= IW, in my experience, I=E2=80=99ve found that =E2=80=9Clist of tuples=E2=80= =9D is really the most likely to be correct way to represent a header block= , because it provides some assurances to the user that the header block has= not been aggressively transformed from how it was sent on the wire. While = the *rules* are that the folded representation is supposed to be semantical= ly equivalent to the unfolded representation, there is nonetheless some inf= ormation implicit in those headers being separate.</div><div><br></div><div= >My intuition when writing this kind of thing is to pass applications (like= Django) the most meaningful representation I can, and then allow the appli= cation to make its own decisions about what meaning they=E2=80=99re willing= to lose. That=E2=80=99s why I=E2=80=99d advocate for =E2=80=9Clist of two-= tuples of bytestrings=E2=80=9D as the representation. However, I don=E2=80= =99t think there=E2=80=99s anything *wrong* with forcing the headers to be = joined by the server where possible: it=E2=80=99s just not how I=E2=80=99d = do it. ;)</div><span><br><blockquote type=3D"cite"><div><div dir=3D"ltr" st= yle=3D"font-family:Helvetica;font-size:12px;font-style:normal;font-weight:n= ormal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform= :none;white-space:normal;word-spacing:0px"><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><div>Set-cookie is the annoying thing here, though. T= hat's why it's dict inbound and list of tuples outbound right now, = and I just don't know if I want to make the inbound one a list of tuple= s too, given I do definitely want to force servers to concat headers togeth= er (unless I find any examples of that screwing things up)</div></div></div= ></div></div></blockquote><br></span></div><div>You could make the inbound = one a list of tuples but still require that the servers concat headers. The= rule then would be that it needs to be possible for an application to say = `dict(headers)` without any loss of meaning.</div></div></blockquote><div><= br></div></span><div>Yes, I think this is a good argument - my worry has al= ways been that the "no multiples" is more of a soft rule that som= e clients might break or some apps might rely on the ordering/multiplicity = of things, so preserving it is _probably_ helpful (and as you say, it lets = the header names go back to bytestrings).</div><div><br></div><div>I'll= modify the spec and then update Daphne and Channels to match; I can leave = Channels parsing both types for a bit, at least.</div><div><br></div><div>C= ollin's point about http2's handling of headers is on point, too - = if the new spec is deliberately thinned down to that point but no further, = it's probably wise to follow them since they know much more about it th= an I do.</div><span><font color=3D"#888888"><div><br></div><div>Andrew=C2= =A0</div></font></span></div></div></div> </blockquote></div><br></div> </div></div><br></div></div><span class=3D"">______________________________= _________________<br> Web-SIG mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]<= /a><br> Web SIG: <a href=3D"http://www.python.org/sigs/web-sig" rel=3D"noreferrer" = target=3D"_blank">http://www.python.org/sigs/web-sig</a><br> Unsubscribe: <a href=3D"https://mail.python.org/mailman/options/web-sig/cma= website%40gmail.com" rel=3D"noreferrer" target=3D"_blank">https://mail.pyth= on.org/mailman/options/web-sig/cmawebsite%40gmail.com</a><br> <br></span></blockquote></div><br></div> </blockquote></div><br></div> --001a113cf5ac9b5c70052dc9c6c5-- --===============4265750724743913954== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Web-SIG mailing list [email protected] Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org --===============4265750724743913954==--