Re: Collating follow-up on the future of WSGI
Benoit Chesneau <[email protected]> Thu, 21 Jan 2016 06:39:51 +0000
| Newsgroups | gmane.comp.python.web |
|---|---|
| Message-ID | <CAJNb-9pp2dwkKQNzFSHe8SsDWed5dQ32-pxx319+ho9f8x-4yQ@mail.gmail.com> |
--===============0822469806727950899== Content-Type: multipart/alternative; boundary=001a114205ba44836d0529d25f76 --001a114205ba44836d0529d25f76 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable because i am not speaking about making a specification, but a way to expose in the API (environ) custom extensions that a server want to experiment. there are actually no easy way except checking "wsgi." indeed but that doesn't make it as clear as a separate namespace where to put all server extensions could be. Like capability field is in imap world. Also I am not trying to force anything, I want to discuss about a possible update of the wsgi spec which I thought was this thread about. What I just want to discuss is the *current* usage of some extensions that have been rapidly dismissed as unworkable. Like I said I will come with a more formal specification about them but I wanted to discuss first about and collect counter arguments which are good too. - beno=C3=AEt On Thu, 21 Jan 2016 at 07:24, Graham Dumpleton <[email protected]> wrote: > I am still confused as to why you keep talking as if you seemingly are > trying to force extensions to the existing WSGI specification into the co= re > WSGI specification when an alternative has already been cited. > > Extensions and the process for describing them and getting them accepted, > plus the appropriate WSGI environ prefix, as has been mentioned before, i= s > what is covered by: > > http://wsgi.readthedocs.org/en/latest/specifications.html > > Being an extension means it is entirely optional for a WSGI server to try > and implement it, thus allowing WSGI servers/adapters that cannot impleme= nt > something to skip them. They stand as separate documents and would never > become part of the core WGSI PEP. > > Is there an issue with doing extensions per the process, and in the WSGI > environ namespace, that was outlined in that URL? You seem to be suggesti= ng > a completely new way of handling extensions and ignoring what was laid do= wn > before. > > So no one is saying you can=E2=80=99t have extensions, and that separate = process > gives you all the scope you need to do it. > > In drafting your specification just fit it reference to what is described > in that URL, using =E2=80=98x-wsgiorg.=E2=80=99 prefix keys. > > Graham > > > On 21 Jan 2016, at 4:13 PM, Benoit Chesneau <[email protected]> wrote: > > I am not speaking about websockets. You could use it for SSE, or some app= s > could use the Upgrade header to upgrade from http to their own protocol > etc... The only discussion i saw about websockets are about the addition = of > an async api or an external api. I am not describing that. I am speaking > about providing a low level abstraction like wsgi.input but adding to it > the support of output. (I was referring to wsgi.multithread...). This low > level interface would allow anyone to provide its own > implementation(server) or usage (application) still acting as a *gateway*= . > > Also who are "we"? I am starting to think the discussion is already done > and only obscure details like the content_length or headers encoding shou= ld > be discussed. The RAW_SOCKET have been added on demand of the gunicorn > users. Such thing also exist in things like cherrypy if I remember. A lot > of code around have been created over it. So before deciding it's > unworkable or whatever I strongly invite you to consider it as an additio= n > to the environ. And since some servers need to pass the data differently = I > then suggest a Resource object on which you can read and write and > eventually poll. This is not a websocket but more a proxy ressource to th= e > client connexion. I will come back asap with a small spec. > > I also propose a second addition to the protocol that formalize the > addition of extensions to the protocol by the servers if they want to. > Having for example something like "`wsg.extensions` . Such addition would > help anyone to experiment changes over the wsgi before making such change= s > in the specification by itself possibly. > > I think we have a good opportunity to extend the WSGI specification to > allow the users to take over the new challenges on the web without forcin= g > them to use a concurrency mode or skip completely the WSGI spec. The > interest I see in WSGI is its simplicity and low level interface allowing > users to build whatever they want over it. The different workers and thei= r > support of different concurrency models and framework in gunicorn let me > think it is possible. Are the participants of this thread ready to discus= s > it? > > - beno=C3=AEt > > On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton <[email protected]= m> > wrote: > >> On 21 Jan 2016, at 9:27 AM, Benoit Chesneau <[email protected]> wrote: >> >> again. any server can do such implementation if we create a new Resource >> abstraction. This abstraction would expose a common api to read and writ= e. >> The implementation would be specific to the server. >> >> >> If you mean not exposing the raw socket and having a separate high level >> API for implementing something like WebSocket this was already talked >> about. The suggestion was that it should not be a part of WSGI. Develop >> that API independently with no link to WSGI. The idea of upgrading from >> WSGI to a different API isn=E2=80=99t practical for various WSGI servers= as it >> isn=E2=80=99t possible to unwind the state of the connection path create= d to get to >> point of handling the WSGI application. The better scenario is that the >> switch to an alternate WebSocket API is handled completely within the we= b >> server however it needs to handle it, when it needs to handle it, and no= t >> be reliant on going into a WSGI application which then says, oh, I actua= lly >> need that to be WebSocket. >> >> Now like we have wsgi.thread I would instead suggest to add a system of >> capability or extension like in smtp, imap, ... so the servers that >> implement a specific extension can legally published it. Would it work f= or >> you? >> >> >> Since there is nothing in WSGI environ called wsgi.thread now I have no >> idea what you are really suggesting here. >> >> Graham >> >> benoit >> >> On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton < >> [email protected]> wrote: >> >>> >>> On 21 Jan 2016, at 2:48 AM, Benoit Chesneau <[email protected]> wrote= : >>> >>> >>> >>> On Wed, Jan 20, 2016 at 1:57 AM Robert Collins < >>> [email protected]> wrote: >>> >>>> On 20 January 2016 at 12:04, Benoit Chesneau <[email protected]> >>>> wrote: >>>> >>>> > >>>> > not at all. But I made the assumption that the wsgi server maintaine= d >>>> a >>>> > thread directly or not where the python application is running . >>>> > >>>> > In any case there is some sort of wrapping done in the same >>>> thread/process >>>> > where the python application is running. And then nothing stop to >>>> give the >>>> > socket away to the application and tell to the server to stop to >>>> communicate >>>> > with it. >>>> >>>> What socket? >>>> >>>> Data could be being passed by shm, for instance. >>>> >>>> -Rob >>>> >>>> >>> While shared memory would be quite a bad idea, then why not. I still >>> don't see why having a way to upgrade the connection can't be done. >>> >>> Call it I/O resource or Socket, the issue is the same. At the end >>> nothing stop the server to pass the control to the app. If we forget th= e >>> socket (which is btw the simplest design) then the server could stop to >>> control the I/O resource when the application ask it to do it. At some >>> point either a garbage collection or a basic resource return/claim flow >>> could be used to definitely free the resource. >>> >>> The thing behind that is that it would allow the WSGI spec to only focu= s >>> on providing a strict gateway workflow without forcing the application = to >>> adopt a concurrency model aync or not. >>> >>> >>> No one has said you cannot do it. because though it is only able to be >>> implemented in a subset of WSGI servers/adapters, then it doesn=E2=80= =99t seem >>> appropriate that it be a part of the core WSGI specification. >>> >>> This is the role of a WSGI extension as found at: >>> >>> http://wsgi.readthedocs.org/en/latest/specifications.html >>> >>> So go talk to the authors of uWSGI, and the other couple of packages >>> available for trying to plug these into some of the pure Python based W= SGI >>> servers and come to an agreement between yourselves as to a standard wa= y of >>> doing it and the extension specification can be added to the wsgi.org >>> site. >>> >>> Graham >>> >>> > --001a114205ba44836d0529d25f76 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div style=3D"white-space:pre-wrap">because i am not speaking about making = a specification, but a way to expose in the API (environ) custom extensions= that a server want to experiment. there are actually no easy way except ch= ecking "wsgi." indeed but that doesn't make it as clear as a = separate namespace where to put all server extensions could be. Like capabi= lity field is in imap world.<br><br>Also I am not trying to force anything,= I want to discuss about a possible update of the wsgi spec which I though= t was this thread about. What I just want to discuss is the *current* usage= of some extensions that have been rapidly dismissed as unworkable. Like I = said I will come with a more formal specification about them but I wanted t= o discuss first about and collect counter arguments which are good too.<br>= <br>- beno=C3=AEt<br><br></div><br><div class=3D"gmail_quote"><div dir=3D"l= tr">On Thu, 21 Jan 2016 at 07:24, Graham Dumpleton <<a href=3D"mailto:gr= [email protected]" target=3D"_blank">[email protected]</a>&= gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0= .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:= break-word">I am still confused as to why you keep talking as if you seemin= gly are trying to force extensions to the existing WSGI specification into = the core WSGI specification when an alternative has already been cited.<div= ><br></div><div>Extensions and the process for describing them and getting = them accepted, plus the appropriate WSGI environ prefix, as has been mentio= ned before, is what is covered by:</div><div><br></div><div>=C2=A0 =C2=A0= =C2=A0<a href=3D"http://wsgi.readthedocs.org/en/latest/specifications.html"= target=3D"_blank">http://wsgi.readthedocs.org/en/latest/specifications.htm= l</a></div><div><br></div><div>Being an extension means it is entirely opti= onal for a WSGI server to try and implement it, thus allowing WSGI servers/= adapters that cannot implement something to skip them. They stand as separa= te documents and would never become part of the core WGSI PEP.</div><div><b= r></div><div>Is there an issue with doing extensions per the process, and i= n the WSGI environ namespace, that was outlined in that URL? You seem to be= suggesting a completely new way of handling extensions and ignoring what w= as laid down before.</div><div><br></div><div>So no one is saying you can= =E2=80=99t have extensions, and that separate process gives you all the sco= pe you need to do it.</div><div><br></div><div>In drafting your specificati= on just fit it reference to what is described in that URL, using =E2=80=98x= -wsgiorg.=E2=80=99 prefix keys.</div><div><br></div><div></div></div><div s= tyle=3D"word-wrap:break-word"><div>Graham</div></div><div style=3D"word-wra= p:break-word"><div><br><div><br><div><blockquote type=3D"cite"><div>On 21 J= an 2016, at 4:13 PM, Benoit Chesneau <<a href=3D"mailto:bchesneau@gmail.= com" target=3D"_blank">[email protected]</a>> wrote:</div><br><div><di= v style=3D"white-space:pre-wrap">I am not speaking about websockets. You c= ould use it for SSE, or some apps could use the Upgrade header to upgrade f= rom http to their own protocol etc... The only discussion i saw about webso= ckets are about the addition of an async api or an external api. I am not d= escribing that. I am speaking about providing a low level abstraction like = wsgi.input but adding to it the support of output. (I was referring to wsgi= .multithread...). This low level interface would allow anyone to provide it= s own implementation(server) or usage (application) still acting as a *gate= way* .<br><br>Also who are "we"? I am starting to think the discu= ssion is already done and only obscure details like the content_length or h= eaders encoding should be discussed. The RAW_SOCKET have been added on dema= nd of the gunicorn users. Such thing also exist in things like cherrypy if = I remember. A lot of code around have been created over it. So before decid= ing it's unworkable or whatever I strongly invite you to consider it as= an addition to the environ. And since some servers need to pass the data d= ifferently I then suggest a Resource object on which you can read and write= and eventually poll. This is not a websocket but more a proxy ressource to= the client connexion. I will come back asap with a small spec.<br><br>I al= so propose a second addition to the protocol that formalize the addition of= extensions to the protocol by the servers if they want to. Having for exam= ple something like "`wsg.extensions` . Such addition would help anyone= to experiment changes over the wsgi before making such changes in the spec= ification by itself possibly.<br><br>I think we have a good opportunity to = extend the WSGI specification to allow the users to take over the new chall= enges on the web without forcing them to use a concurrency mode or skip com= pletely the WSGI spec. The interest I see in WSGI is its simplicity and low= level interface allowing users to build whatever they want over it. The di= fferent workers and their support of different concurrency models and frame= work in gunicorn let me think it is possible. Are the participants of this= thread ready to discuss it? <br><br>- beno=C3=AEt</div><br><div class=3D"g= mail_quote"><div dir=3D"ltr">On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton= <<a href=3D"mailto:[email protected]" target=3D"_blank">graham= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quo= te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"= ><div style=3D"word-wrap:break-word"><div><blockquote type=3D"cite"><div>On= 21 Jan 2016, at 9:27 AM, Benoit Chesneau <<a href=3D"mailto:bchesneau@g= mail.com" target=3D"_blank">[email protected]</a>> wrote:</div><br><di= v><div style=3D"white-space:pre-wrap">again. any server can do such impleme= ntation if we create a new Resource abstraction. This abstraction would exp= ose a common api to read and write. The implementation would be specific to= the server.<br></div></div></blockquote><div><br></div></div></div><div st= yle=3D"word-wrap:break-word"><div><div>If you mean not exposing the raw soc= ket and having a separate high level API for implementing something like We= bSocket this was already talked about. The suggestion was that it should no= t be a part of WSGI. Develop that API independently with no link to WSGI. T= he idea of upgrading from WSGI to a different API isn=E2=80=99t practical f= or various WSGI servers as it isn=E2=80=99t possible to unwind the state of= the connection path created to get to point of handling the WSGI applicati= on. The better scenario is that the switch to an alternate WebSocket API is= handled completely within the web server however it needs to handle it, wh= en it needs to handle it, and not be reliant on going into a WSGI applicati= on which then says, oh, I actually need that to be WebSocket.</div></div></= div><div style=3D"word-wrap:break-word"><div><br><blockquote type=3D"cite">= <div><div style=3D"white-space:pre-wrap">Now like we have wsgi.thread I wou= ld instead suggest to add a system of capability or extension like in smtp= , imap, ... so the servers that implement a specific extension can legally = published it. Would it work for you?<br></div></div></blockquote><div><br><= /div></div></div><div style=3D"word-wrap:break-word"><div><div>Since there = is nothing in WSGI environ called wsgi.thread now I have no idea what you a= re really suggesting here.</div></div></div><div style=3D"word-wrap:break-w= ord"><div><div><br></div><div>Graham</div></div></div><div style=3D"word-wr= ap:break-word"><div><br><blockquote type=3D"cite"><div><div style=3D"white-= space:pre-wrap">benoit</div><br><div class=3D"gmail_quote"><div dir=3D"ltr"= >On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton <<a href=3D"mailto:graha= [email protected]" target=3D"_blank">[email protected]</a>>= wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8= ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:bre= ak-word"><br><div><blockquote type=3D"cite"><div>On 21 Jan 2016, at 2:48 AM= , Benoit Chesneau <<a href=3D"mailto:[email protected]" target=3D"_bla= nk">[email protected]</a>> wrote:</div><br><div><div dir=3D"ltr" style= =3D"font-family:Helvetica;font-size:16px;font-style:normal;font-variant:nor= mal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0= px;text-transform:none;white-space:normal;word-spacing:0px"><br><br><div cl= ass=3D"gmail_quote"><div dir=3D"ltr">On Wed, Jan 20, 2016 at 1:57 AM Robert= Collins <<a href=3D"mailto:[email protected]" target=3D"_blank"= >[email protected]</a>> wrote:<br></div><blockquote class=3D"gma= il_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-le= ft-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">On 20 J= anuary 2016 at 12:04, Benoit Chesneau <<a href=3D"mailto:bchesneau@gmail= .com" target=3D"_blank">[email protected]</a>> wrote:<br><br>><br>&= gt; not at all. But I made the assumption that the wsgi server maintained a= <br>> thread directly or not where the python application is running .<b= r>><br>> In any case there is some sort of wrapping done in the same = thread/process<br>> where the python application is running. And then no= thing stop to give the<br>> socket away to the application and tell to t= he server to stop to communicate<br>> with it.<br><br>What socket?<br><b= r>Data could be being passed by shm, for instance.<br><br>-Rob<br><br></blo= ckquote><div><br></div><div>While shared memory would be quite a bad idea, = then why not. I still don't see why having a way to upgrade the connect= ion can't be done.</div><div><br></div><div>Call it I/O resource or Soc= ket, the issue is the same. At the end nothing stop the server to pass the = control to the app. If we forget the socket (which is btw the simplest desi= gn) then the server could stop to control the I/O resource when the applica= tion ask it to do it. At some point either a garbage collection or a basic = resource return/claim flow could be used to definitely free the resource.<b= r></div><div><br></div><div>The thing behind that is that it would allow th= e WSGI spec to only focus on providing a strict gateway workflow without fo= rcing the application to adopt a concurrency model aync or not.</div></div>= </div></div></blockquote><br></div></div><div style=3D"word-wrap:break-word= "><div>No one has said you cannot do it. because though it is only able to = be implemented in a subset of WSGI servers/adapters, then it doesn=E2=80=99= t seem appropriate that it be a part of the core WSGI specification.</div><= div><br></div><div>This is the role of a WSGI extension as found at:</div><= div><br></div><div>=C2=A0 =C2=A0=C2=A0<a href=3D"http://wsgi.readthedocs.or= g/en/latest/specifications.html" target=3D"_blank">http://wsgi.readthedocs.= org/en/latest/specifications.html</a></div><div><br></div><div>So go talk t= o the authors of uWSGI, and the other couple of packages available for tryi= ng to plug these into some of the pure Python based WSGI servers and come t= o an agreement between yourselves as to a standard way of doing it and the = extension specification can be added to the <a href=3D"http://wsgi.org/" ta= rget=3D"_blank">wsgi.org</a>=C2=A0site.</div></div><div style=3D"word-wrap:= break-word"><div><br></div><div>Graham</div><br></div></blockquote></div> </div></blockquote></div></div></blockquote></div> </div></blockquote></div><br></div></div></div></blockquote></div> --001a114205ba44836d0529d25f76-- --===============0822469806727950899== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Web-SIG mailing list [email protected] Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org --===============0822469806727950899==--