Re: Collating follow-up on the future of WSGI
Benoit Chesneau <[email protected]> Thu, 21 Jan 2016 05:13:51 +0000
| Newsgroups | gmane.comp.python.web |
|---|---|
| Message-ID | <CAJNb-9pT-fKHdrp7SA40YgyA7rKGixne4NDAXHD41DB6r9wyww@mail.gmail.com> |
--===============5699942482635566044== Content-Type: multipart/alternative; boundary=001a11c3eebca984490529d12bfa --001a11c3eebca984490529d12bfa Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I am not speaking about websockets. You could use it for SSE, or some apps could use the Upgrade header to upgrade from http to their own protocol etc... The only discussion i saw about websockets are about the addition of an async api or an external api. I am not describing that. I am speaking about providing a low level abstraction like wsgi.input but adding to it the support of output. (I was referring to wsgi.multithread...). This low level interface would allow anyone to provide its own implementation(server) or usage (application) still acting as a *gateway* . Also who are "we"? I am starting to think the discussion is already done and only obscure details like the content_length or headers encoding should be discussed. The RAW_SOCKET have been added on demand of the gunicorn users. Such thing also exist in things like cherrypy if I remember. A lot of code around have been created over it. So before deciding it's unworkable or whatever I strongly invite you to consider it as an addition to the environ. And since some servers need to pass the data differently I then suggest a Resource object on which you can read and write and eventually poll. This is not a websocket but more a proxy ressource to the client connexion. I will come back asap with a small spec. I also propose a second addition to the protocol that formalize the addition of extensions to the protocol by the servers if they want to. Having for example something like "`wsg.extensions` . Such addition would help anyone to experiment changes over the wsgi before making such changes in the specification by itself possibly. I think we have a good opportunity to extend the WSGI specification to allow the users to take over the new challenges on the web without forcing them to use a concurrency mode or skip completely the WSGI spec. The interest I see in WSGI is its simplicity and low level interface allowing users to build whatever they want over it. The different workers and their support of different concurrency models and framework in gunicorn let me think it is possible. Are the participants of this thread ready to discuss it? - beno=C3=AEt On Wed, 20 Jan 2016 at 23:37, Graham Dumpleton <[email protected]> wrote: > On 21 Jan 2016, at 9:27 AM, Benoit Chesneau <[email protected]> wrote: > > again. any server can do such implementation if we create a new Resource > abstraction. This abstraction would expose a common api to read and write= . > The implementation would be specific to the server. > > > If you mean not exposing the raw socket and having a separate high level > API for implementing something like WebSocket this was already talked > about. The suggestion was that it should not be a part of WSGI. Develop > that API independently with no link to WSGI. The idea of upgrading from > WSGI to a different API isn=E2=80=99t practical for various WSGI servers = as it > isn=E2=80=99t possible to unwind the state of the connection path created= to get to > point of handling the WSGI application. The better scenario is that the > switch to an alternate WebSocket API is handled completely within the web > server however it needs to handle it, when it needs to handle it, and not > be reliant on going into a WSGI application which then says, oh, I actual= ly > need that to be WebSocket. > > Now like we have wsgi.thread I would instead suggest to add a system of > capability or extension like in smtp, imap, ... so the servers that > implement a specific extension can legally published it. Would it work fo= r > you? > > > Since there is nothing in WSGI environ called wsgi.thread now I have no > idea what you are really suggesting here. > > Graham > > benoit > > On Wed, 20 Jan 2016 at 21:28, Graham Dumpleton <[email protected]= m> > wrote: > >> >> On 21 Jan 2016, at 2:48 AM, Benoit Chesneau <[email protected]> wrote: >> >> >> >> On Wed, Jan 20, 2016 at 1:57 AM Robert Collins <[email protected]= t> >> wrote: >> >>> On 20 January 2016 at 12:04, Benoit Chesneau <[email protected]> >>> wrote: >>> >>> > >>> > not at all. But I made the assumption that the wsgi server maintained= a >>> > thread directly or not where the python application is running . >>> > >>> > In any case there is some sort of wrapping done in the same >>> thread/process >>> > where the python application is running. And then nothing stop to giv= e >>> the >>> > socket away to the application and tell to the server to stop to >>> communicate >>> > with it. >>> >>> What socket? >>> >>> Data could be being passed by shm, for instance. >>> >>> -Rob >>> >>> >> While shared memory would be quite a bad idea, then why not. I still >> don't see why having a way to upgrade the connection can't be done. >> >> Call it I/O resource or Socket, the issue is the same. At the end nothin= g >> stop the server to pass the control to the app. If we forget the socket >> (which is btw the simplest design) then the server could stop to control >> the I/O resource when the application ask it to do it. At some point eit= her >> a garbage collection or a basic resource return/claim flow could be used= to >> definitely free the resource. >> >> The thing behind that is that it would allow the WSGI spec to only focus >> on providing a strict gateway workflow without forcing the application t= o >> adopt a concurrency model aync or not. >> >> >> No one has said you cannot do it. because though it is only able to be >> implemented in a subset of WSGI servers/adapters, then it doesn=E2=80=99= t seem >> appropriate that it be a part of the core WSGI specification. >> >> This is the role of a WSGI extension as found at: >> >> http://wsgi.readthedocs.org/en/latest/specifications.html >> >> So go talk to the authors of uWSGI, and the other couple of packages >> available for trying to plug these into some of the pure Python based WS= GI >> servers and come to an agreement between yourselves as to a standard way= of >> doing it and the extension specification can be added to the wsgi.org >> site. >> >> Graham >> >> --001a11c3eebca984490529d12bfa Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div style=3D"white-space:pre-wrap">I am not speaking about websockets. Yo= u could use it for SSE, or some apps could use the Upgrade header to upgrad= e from http to their own protocol etc... The only discussion i saw about we= bsockets are about the addition of an async api or an external api. I am no= t describing that. I am speaking about providing a low level abstraction li= ke wsgi.input but adding to it the support of output. (I was referring to w= sgi.multithread...). This low level interface would allow anyone to provide= its own implementation(server) or usage (application) still acting as a *g= ateway* .<br><br>Also who are "we"? I am starting to think the di= scussion is already done and only obscure details like the content_length o= r headers encoding should be discussed. The RAW_SOCKET have been added on d= emand of the gunicorn users. Such thing also exist in things like cherrypy = if I remember. A lot of code around have been created over it. So before de= ciding it's unworkable or whatever I strongly invite you to consider it= as an addition to the environ. And since some servers need to pass the dat= a differently I then suggest a Resource object on which you can read and wr= ite and eventually poll. This is not a websocket but more a proxy ressource= to the client connexion. I will come back asap with a small spec.<br><br>I= also propose a second addition to the protocol that formalize the addition= of extensions to the protocol by the servers if they want to. Having for e= xample something like "`wsg.extensions` . Such addition would help any= one to experiment changes over the wsgi before making such changes in the s= pecification by itself possibly.<br><br>I think we have a good opportunity = to extend the WSGI specification to allow the users to take over the new ch= allenges on the web without forcing them to use a concurrency mode or skip = completely the WSGI spec. The interest I see in WSGI is its simplicity and = low level interface allowing users to build whatever they want over it. The= different workers and their support of different concurrency models and fr= amework in gunicorn let me think it is possible. Are the participants of t= his thread ready to discuss it? <br><br>- beno=C3=AEt</div><br><div class= =3D"gmail_quote"><div dir=3D"ltr">On Wed, 20 Jan 2016 at 23:37, Graham Dump= leton <<a href=3D"mailto:[email protected]">graham.dumpleton@gm= ail.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"= margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style= =3D"word-wrap:break-word"><div><blockquote type=3D"cite"><div>On 21 Jan 201= 6, at 9:27 AM, Benoit Chesneau <<a href=3D"mailto:[email protected]" t= arget=3D"_blank">[email protected]</a>> wrote:</div><br><div><div styl= e=3D"white-space:pre-wrap">again. any server can do such implementation if = we create a new Resource abstraction. This abstraction would expose a commo= n api to read and write. The implementation would be specific to the server= .<br></div></div></blockquote><div><br></div></div></div><div style=3D"word= -wrap:break-word"><div><div>If you mean not exposing the raw socket and hav= ing a separate high level API for implementing something like WebSocket thi= s was already talked about. The suggestion was that it should not be a part= of WSGI. Develop that API independently with no link to WSGI. The idea of = upgrading from WSGI to a different API isn=E2=80=99t practical for various = WSGI servers as it isn=E2=80=99t possible to unwind the state of the connec= tion path created to get to point of handling the WSGI application. The bet= ter scenario is that the switch to an alternate WebSocket API is handled co= mpletely within the web server however it needs to handle it, when it needs= to handle it, and not be reliant on going into a WSGI application which th= en says, oh, I actually need that to be WebSocket.</div></div></div><div st= yle=3D"word-wrap:break-word"><div><br><blockquote type=3D"cite"><div><div s= tyle=3D"white-space:pre-wrap">Now like we have wsgi.thread I would instead = suggest to add a system of capability or extension like in smtp, imap, ...= so the servers that implement a specific extension can legally published i= t. Would it work for you?<br></div></div></blockquote><div><br></div></div>= </div><div style=3D"word-wrap:break-word"><div><div>Since there is nothing = in WSGI environ called wsgi.thread now I have no idea what you are really s= uggesting here.</div></div></div><div style=3D"word-wrap:break-word"><div><= div><br></div><div>Graham</div></div></div><div style=3D"word-wrap:break-wo= rd"><div><br><blockquote type=3D"cite"><div><div style=3D"white-space:pre-w= rap">benoit</div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed, 20= Jan 2016 at 21:28, Graham Dumpleton <<a href=3D"mailto:graham.dumpleton= @gmail.com" target=3D"_blank">[email protected]</a>> wrote:<br>= </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-word"><b= r><div><blockquote type=3D"cite"><div>On 21 Jan 2016, at 2:48 AM, Benoit Ch= esneau <<a href=3D"mailto:[email protected]" target=3D"_blank">bchesne= [email protected]</a>> wrote:</div><br><div><div dir=3D"ltr" style=3D"font-fa= mily:Helvetica;font-size:16px;font-style:normal;font-variant:normal;font-we= ight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-tra= nsform:none;white-space:normal;word-spacing:0px"><br><br><div class=3D"gmai= l_quote"><div dir=3D"ltr">On Wed, Jan 20, 2016 at 1:57 AM Robert Collins &l= t;<a href=3D"mailto:[email protected]" target=3D"_blank">robertc@ro= bertcollins.net</a>> wrote:<br></div><blockquote class=3D"gmail_quote" s= tyle=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rg= b(204,204,204);border-left-style:solid;padding-left:1ex">On 20 January 2016= at 12:04, Benoit Chesneau <<a href=3D"mailto:[email protected]" targe= t=3D"_blank">[email protected]</a>> wrote:<br><br>><br>> not at = all. But I made the assumption that the wsgi server maintained a<br>> th= read directly or not where the python application is running .<br>><br>&= gt; In any case there is some sort of wrapping done in the same thread/proc= ess<br>> where the python application is running. And then nothing stop = to give the<br>> socket away to the application and tell to the server t= o stop to communicate<br>> with it.<br><br>What socket?<br><br>Data coul= d be being passed by shm, for instance.<br><br>-Rob<br><br></blockquote><di= v><br></div><div>While shared memory would be quite a bad idea, then why no= t. I still don't see why having a way to upgrade the connection can'= ;t be done.</div><div><br></div><div>Call it I/O resource or Socket, the is= sue is the same. At the end nothing stop the server to pass the control to = the app. If we forget the socket (which is btw the simplest design) then th= e server could stop to control the I/O resource when the application ask it= to do it. At some point either a garbage collection or a basic resource re= turn/claim flow could be used to definitely free the resource.<br></div><di= v><br></div><div>The thing behind that is that it would allow the WSGI spec= to only focus on providing a strict gateway workflow without forcing the a= pplication to adopt a concurrency model aync or not.</div></div></div></div= ></blockquote><br></div></div><div style=3D"word-wrap:break-word"><div>No o= ne has said you cannot do it. because though it is only able to be implemen= ted in a subset of WSGI servers/adapters, then it doesn=E2=80=99t seem appr= opriate that it be a part of the core WSGI specification.</div><div><br></d= iv><div>This is the role of a WSGI extension as found at:</div><div><br></d= iv><div>=C2=A0 =C2=A0=C2=A0<a href=3D"http://wsgi.readthedocs.org/en/latest= /specifications.html" target=3D"_blank">http://wsgi.readthedocs.org/en/late= st/specifications.html</a></div><div><br></div><div>So go talk to the autho= rs of uWSGI, and the other couple of packages available for trying to plug = these into some of the pure Python based WSGI servers and come to an agreem= ent between yourselves as to a standard way of doing it and the extension s= pecification can be added to the <a href=3D"http://wsgi.org/" target=3D"_bl= ank">wsgi.org</a>=C2=A0site.</div></div><div style=3D"word-wrap:break-word"= ><div><br></div><div>Graham</div><br></div></blockquote></div> </div></blockquote></div></div></blockquote></div> --001a11c3eebca984490529d12bfa-- --===============5699942482635566044== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Web-SIG mailing list [email protected] Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org --===============5699942482635566044==--