Fwd: Collating follow-up on the future of WSGI
Benoit Chesneau <[email protected]> Thu, 21 Jan 2016 05:12:51 +0000
| Newsgroups | gmane.comp.python.web |
|---|---|
| Message-ID | <CAJNb-9rwkppcYrXRmdT7djdaCS1N10L+J_BRzskfKy=Er4c_2Q@mail.gmail.com> |
--===============3342396397026852345== Content-Type: multipart/alternative; boundary=001a114187c41408990529d1289d --001a114187c41408990529d1289d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable ---------- Forwarded message --------- From: Benoit Chesneau <[email protected]> Date: Thu, 21 Jan 2016 at 06:12 Subject: Re: [Web-SIG] Collating follow-up on the future of WSGI To: Graham Dumpleton <[email protected]> I am not speaking about websockets. You could use it for SSE, or some apps could use the Upgrade header to upgrade from http to their own protocol etc... The only discussion i saw about websockets are about the addition of an async api or an external api. I am not describing that. I am speaking about providing a low level abstraction like wsgi.input but adding to it the support of output. (I was referring to wsgi.multithread...). This low level interface would allow anyone to provide its own implementation(server) or usage (application) still acting as a *gateway* . Also who are "we"? I am starting to think the discussion is already done and only obscure details like the content_length or headers encoding should be discussed. The RAW_SOCKET have been added on demand of the gunicorn users. Such thing also exist in things like cherrypy if I remember. A lot of code around have been created over it. So before deciding it's unworkable or whatever I strongly invite you to consider it as an addition to the environ. And since some servers need to pass the data differently I then suggest a Resource object on which you can read and write and eventually poll. This is not a websocket but more a proxy ressource to the client connexion. I will come back asap with a small spec. I also propose a second addition to the protocol that formalize the addition of extensions to the protocol by the servers if they want to. Having for example something like "`wsg.extensions` . Such addition would help anyone to experiment changes over the wsgi before making such changes in the specification by itself possibly. I think we have a good opportunity to extend the WSGI specification to allow the users to take over the new challenges on the web without forcing them to use a concurrency mode or skip completely the WSGI spec. The interest I see in WSGI is its simplicity and low level interface allowing users to build whatever they want over it. The different workers and their support of different concurrency models and framework in gunicorn let me think it is possible. Are the participants of this thread ready to discuss it? - beno=C3=AEt On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton <[email protected]> wrote: > On 21 Jan 2016, at 9:27 AM, Benoit Chesneau <[email protected]> wrote: > > again. any server can do such implementation if we create a new Resource > abstraction. This abstraction would expose a common api to read and write= . > The implementation would be specific to the server. > > > If you mean not exposing the raw socket and having a separate high level > API for implementing something like WebSocket this was already talked > about. The suggestion was that it should not be a part of WSGI. Develop > that API independently with no link to WSGI. The idea of upgrading from > WSGI to a different API isn=E2=80=99t practical for various WSGI servers = as it > isn=E2=80=99t possible to unwind the state of the connection path created= to get to > point of handling the WSGI application. The better scenario is that the > switch to an alternate WebSocket API is handled completely within the web > server however it needs to handle it, when it needs to handle it, and not > be reliant on going into a WSGI application which then says, oh, I actual= ly > need that to be WebSocket. > > Now like we have wsgi.thread I would instead suggest to add a system of > capability or extension like in smtp, imap, ... so the servers that > implement a specific extension can legally published it. Would it work fo= r > you? > > > Since there is nothing in WSGI environ called wsgi.thread now I have no > idea what you are really suggesting here. > > Graham > > benoit > > On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton <[email protected]= m> > wrote: > >> >> On 21 Jan 2016, at 2:48 AM, Benoit Chesneau <[email protected]> wrote: >> >> >> >> On Wed, Jan 20, 2016 at 1:57 AM Robert Collins <[email protected]= t> >> wrote: >> >>> On 20 January 2016 at 12:04, Benoit Chesneau <[email protected]> >>> wrote: >>> >>> > >>> > not at all. But I made the assumption that the wsgi server maintained= a >>> > thread directly or not where the python application is running . >>> > >>> > In any case there is some sort of wrapping done in the same >>> thread/process >>> > where the python application is running. And then nothing stop to giv= e >>> the >>> > socket away to the application and tell to the server to stop to >>> communicate >>> > with it. >>> >>> What socket? >>> >>> Data could be being passed by shm, for instance. >>> >>> -Rob >>> >>> >> While shared memory would be quite a bad idea, then why not. I still >> don't see why having a way to upgrade the connection can't be done. >> >> Call it I/O resource or Socket, the issue is the same. At the end nothin= g >> stop the server to pass the control to the app. If we forget the socket >> (which is btw the simplest design) then the server could stop to control >> the I/O resource when the application ask it to do it. At some point eit= her >> a garbage collection or a basic resource return/claim flow could be used= to >> definitely free the resource. >> >> The thing behind that is that it would allow the WSGI spec to only focus >> on providing a strict gateway workflow without forcing the application t= o >> adopt a concurrency model aync or not. >> >> >> No one has said you cannot do it. because though it is only able to be >> implemented in a subset of WSGI servers/adapters, then it doesn=E2=80=99= t seem >> appropriate that it be a part of the core WSGI specification. >> >> This is the role of a WSGI extension as found at: >> >> http://wsgi.readthedocs.org/en/latest/specifications.html >> >> So go talk to the authors of uWSGI, and the other couple of packages >> available for trying to plug these into some of the pure Python based WS= GI >> servers and come to an agreement between yourselves as to a standard way= of >> doing it and the extension specification can be added to the wsgi.org >> site. >> >> Graham >> >> --001a114187c41408990529d1289d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div style=3D"white-space:pre-wrap"></div><br><div class=3D"gmail_quote"><d= iv dir=3D"ltr">---------- Forwarded message ---------<br>From: Benoit Chesn= eau <<a href=3D"mailto:[email protected]">[email protected]</a>><= br>Date: Thu, 21 Jan 2016 at 06:12<br>Subject: Re: [Web-SIG] Collating foll= ow-up on the future of WSGI<br>To: Graham Dumpleton <<a href=3D"mailto:g= [email protected]">[email protected]</a>><br></div><br>= <br><div style=3D"white-space:pre-wrap">I am not speaking about websockets.= You could use it for SSE, or some apps could use the Upgrade header to up= grade from http to their own protocol etc... The only discussion i saw abou= t websockets are about the addition of an async api or an external api. I a= m not describing that. I am speaking about providing a low level abstractio= n like wsgi.input but adding to it the support of output. (I was referring = to wsgi.multithread...). This low level interface would allow anyone to pro= vide its own implementation(server) or usage (application) still acting as = a *gateway* .<br><br>Also who are "we"? I am starting to think th= e discussion is already done and only obscure details like the content_leng= th or headers encoding should be discussed. The RAW_SOCKET have been added = on demand of the gunicorn users. Such thing also exist in things like cherr= ypy if I remember. A lot of code around have been created over it. So befor= e deciding it's unworkable or whatever I strongly invite you to conside= r it as an addition to the environ. And since some servers need to pass the= data differently I then suggest a Resource object on which you can read an= d write and eventually poll. This is not a websocket but more a proxy resso= urce to the client connexion. I will come back asap with a small spec.<br><= br>I also propose a second addition to the protocol that formalize the addi= tion of extensions to the protocol by the servers if they want to. Having f= or example something like "`wsg.extensions` . Such addition would help= anyone to experiment changes over the wsgi before making such changes in t= he specification by itself possibly.<br><br>I think we have a good opportun= ity to extend the WSGI specification to allow the users to take over the ne= w challenges on the web without forcing them to use a concurrency mode or s= kip completely the WSGI spec. The interest I see in WSGI is its simplicity = and low level interface allowing users to build whatever they want over it.= The different workers and their support of different concurrency models an= d framework in gunicorn let me think it is possible. Are the participants = of this thread ready to discuss it? <br><br>- beno=C3=AEt<br><br></div><br>= <div class=3D"gmail_quote"><div dir=3D"ltr">On Wed, 20 Jan 2016 at 23:37, G= raham Dumpleton <<a href=3D"mailto:[email protected]" target=3D= "_blank">[email protected]</a>> wrote:<br></div><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa= dding-left:1ex"><div style=3D"word-wrap:break-word"><div><blockquote type= =3D"cite"><div>On 21 Jan 2016, at 9:27 AM, Benoit Chesneau <<a href=3D"m= ailto:[email protected]" target=3D"_blank">[email protected]</a>> wr= ote:</div><br><div><div style=3D"white-space:pre-wrap">again. any server ca= n do such implementation if we create a new Resource abstraction. This abst= raction would expose a common api to read and write. The implementation wou= ld be specific to the server.<br></div></div></blockquote><div><br></div></= div></div><div style=3D"word-wrap:break-word"><div><div>If you mean not exp= osing the raw socket and having a separate high level API for implementing = something like WebSocket this was already talked about. The suggestion was = that it should not be a part of WSGI. Develop that API independently with n= o link to WSGI. The idea of upgrading from WSGI to a different API isn=E2= =80=99t practical for various WSGI servers as it isn=E2=80=99t possible to = unwind the state of the connection path created to get to point of handling= the WSGI application. The better scenario is that the switch to an alterna= te WebSocket API is handled completely within the web server however it nee= ds to handle it, when it needs to handle it, and not be reliant on going in= to a WSGI application which then says, oh, I actually need that to be WebSo= cket.</div></div></div><div style=3D"word-wrap:break-word"><div><br><blockq= uote type=3D"cite"><div><div style=3D"white-space:pre-wrap">Now like we hav= e wsgi.thread I would instead suggest to add a system of capability or exte= nsion like in smtp, imap, ... so the servers that implement a specific ext= ension can legally published it. Would it work for you?<br></div></div></bl= ockquote><div><br></div></div></div><div style=3D"word-wrap:break-word"><di= v><div>Since there is nothing in WSGI environ called wsgi.thread now I have= no idea what you are really suggesting here.</div></div></div><div style= =3D"word-wrap:break-word"><div><div><br></div><div>Graham</div></div></div>= <div style=3D"word-wrap:break-word"><div><br><blockquote type=3D"cite"><div= ><div style=3D"white-space:pre-wrap">benoit</div><br><div class=3D"gmail_qu= ote"><div dir=3D"ltr">On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton <<a= href=3D"mailto:[email protected]" target=3D"_blank">graham.dumple= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote" sty= le=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div s= tyle=3D"word-wrap:break-word"><br><div><blockquote type=3D"cite"><div>On 21= Jan 2016, at 2:48 AM, Benoit Chesneau <<a href=3D"mailto:bchesneau@gmai= l.com" target=3D"_blank">[email protected]</a>> wrote:</div><br><div><= div dir=3D"ltr" style=3D"font-family:Helvetica;font-size:16px;font-style:no= rmal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-alig= n:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing= :0px"><br><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed, Jan 20, 2= 016 at 1:57 AM Robert Collins <<a href=3D"mailto:[email protected]= et" target=3D"_blank">[email protected]</a>> wrote:<br></div><bl= ockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-lef= t-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padd= ing-left:1ex">On 20 January 2016 at 12:04, Benoit Chesneau <<a href=3D"m= ailto:[email protected]" target=3D"_blank">[email protected]</a>> wr= ote:<br><br>><br>> not at all. But I made the assumption that the wsg= i server maintained a<br>> thread directly or not where the python appli= cation is running .<br>><br>> In any case there is some sort of wrapp= ing done in the same thread/process<br>> where the python application is= running. And then nothing stop to give the<br>> socket away to the appl= ication and tell to the server to stop to communicate<br>> with it.<br><= br>What socket?<br><br>Data could be being passed by shm, for instance.<br>= <br>-Rob<br><br></blockquote><div><br></div><div>While shared memory would = be quite a bad idea, then why not. I still don't see why having a way t= o upgrade the connection can't be done.</div><div><br></div><div>Call i= t I/O resource or Socket, the issue is the same. At the end nothing stop th= e server to pass the control to the app. If we forget the socket (which is = btw the simplest design) then the server could stop to control the I/O reso= urce when the application ask it to do it. At some point either a garbage c= ollection or a basic resource return/claim flow could be used to definitely= free the resource.<br></div><div><br></div><div>The thing behind that is t= hat it would allow the WSGI spec to only focus on providing a strict gatewa= y workflow without forcing the application to adopt a concurrency model ayn= c or not.</div></div></div></div></blockquote><br></div></div><div style=3D= "word-wrap:break-word"><div>No one has said you cannot do it. because thoug= h it is only able to be implemented in a subset of WSGI servers/adapters, t= hen it doesn=E2=80=99t seem appropriate that it be a part of the core WSGI = specification.</div><div><br></div><div>This is the role of a WSGI extensio= n as found at:</div><div><br></div><div>=C2=A0 =C2=A0=C2=A0<a href=3D"http:= //wsgi.readthedocs.org/en/latest/specifications.html" target=3D"_blank">htt= p://wsgi.readthedocs.org/en/latest/specifications.html</a></div><div><br></= div><div>So go talk to the authors of uWSGI, and the other couple of packag= es available for trying to plug these into some of the pure Python based WS= GI servers and come to an agreement between yourselves as to a standard way= of doing it and the extension specification can be added to the <a href=3D= "http://wsgi.org/" target=3D"_blank">wsgi.org</a>=C2=A0site.</div></div><di= v style=3D"word-wrap:break-word"><div><br></div><div>Graham</div><br></div>= </blockquote></div> </div></blockquote></div></div></blockquote></div></div> --001a114187c41408990529d1289d-- --===============3342396397026852345== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Web-SIG mailing list [email protected] Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org --===============3342396397026852345==--