Re: sharing a dict between child processes
Sean DiZazzo <[email protected]> Wed, 6 Nov 2019 21:29:34 -0800
| Newsgroups | gmane.comp.python.twisted |
|---|---|
| Message-ID | <CANzH6evFWU5i_97t-uHUcziwbh9WKvt06Sd30496OPjbUpfP4g@mail.gmail.com> |
--===============3567177367930515327== Content-Type: multipart/alternative; boundary="000000000000034e110596baf405" --000000000000034e110596baf405 Content-Type: text/plain; charset="UTF-8" If you need guaranteed delivery of the data, why not just use a TCP connection to the unix socket, instead of a UDP connection which inherently can lose data? In that case I don't think your patch would be needed. I didn't look at the source, so perhaps I missed something. On Wed, Nov 6, 2019 at 9:10 AM Scott, Barry <[email protected]> wrote: > On Wednesday, 6 November 2019 16:43:52 GMT Waqar Khan wrote: > > Hi Barry, > > Thanks for the response. Where can I read more about (1). It > seems > > like that is something I need to explore. > > As we already have (2) (cache for each process). > > Thanks again for your help. > > We use the UDS (Unix domain sockets) to talk to a master process. > Twisted has support for this. But you need a small patch to avoid data > lose. > > UDS does not lose data and is message based, not bytes based. We > use pickle to encode requests and responses. > > Barry > > The patch is: > > --- Twisted-18.4.0.orig/src/twisted/internet/unix.py.orig 2018-08-01 > 12:45:38.711115425 +0100 > +++ Twisted-18.4.0/src/twisted/internet/unix.py 2018-08-01 > 12:45:47.946115123 > +0100 > @@ -509,11 +509,6 @@ > return self.write(datagram, address) > elif no == EMSGSIZE: > raise error.MessageLengthError("message too long") > - elif no == EAGAIN: > - # oh, well, drop the data. The only difference from UDP > - # is that UDP won't ever notice. > - # TODO: add TCP-like buffering > - pass > else: > raise > > You then have to handle the EAGAIN error and do retries yourself. > As it stands the patch is not good enough to put into twisted as a > full fix would need to put the handling of the retries into twisted. > > I guess (2) does not work for you as the cache hit rate is low > and you need to share the cache to get a benefit. Cache entries > only get used a few times? > > In our case the hit rate is high (99%+) and we just pay the cost of > populating the caches on process start up, which ends up being > noise. > > Barry > > > > > On Wed, Nov 6, 2019 at 8:39 AM Scott, Barry <[email protected]> > > > > wrote: > > > On Wednesday, 6 November 2019 14:21:22 GMT Maarten ter Huurne wrote: > > > > On Wednesday, 6 November 2019 07:19:56 CET Waqar Khan wrote: > > > > > Hi, > > > > > So, I am writing a twisted server. This server spawn multiple child > > > > > processes using reactor spawnProcess that initializes a process > > > > > protocol. > > > > > > > > > > Now, each of the childprocess receives some REST requests. Each > > > > > process has a dict that acts as cache. > > > > > Now, I want to share dict across processes. > > > > > In general, python has SharedMemoryManager in multiprocessing > module > > > > > which would have helped. > > > > > > https://docs.python.org/3/library/multiprocessing.shared_memory.html#m > > > > > ultiprocessing.managers.SharedMemoryManager.SharedMemory But since > I > > > > > am using twisted internal process implementation, how do I share > this > > > > > dict across the processes so that all the processes use this common > > > > > cache? > > > > > > > > Keeping a dictionary in SharedMemoryManager seems far from trivial. I > > > > don't think you can allocate arbitrary Python objects in the shared > > > > memory and even if you could, you would run into problems when one > > > > process mutates the dictionary while another is looking up something > or > > > > also mutating it. > > > > > > > > It could in theory work if you implement a custom lock-less > dictionary, > > > > but that would be a lot of work and hard to get right. Also having > > > > shared memory mutations be synced between multiple CPU cores could > > > > degrade performance, since keeping core-local CPU caches in sync is > > > > expensive. > > > > > > > > Would it be an option to have only one process accept the REST > requests, > > > > check whether the result is in the cache and only distribute work to > the > > > > other processes if you get a cache miss? Typically the case where an > > > > answer is cached is pretty fast, so perhaps you don't need multiple > > > > processes to handle incoming requests. > > > > > > We have used a couple of ways to cache. > > > 1. Use a singleton process to hold the cache and ask it, via IPC, for > > > answers > > > from the other process. > > > 2. have a cache in each process > > > > > > Barry > > > > > > > Bye, > > > > > > > > Maarten > > > > > > > > _______________________________________________ > > > > Twisted-Python mailing list > > > > [email protected] > > > > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > > > > _______________________________________________ > > > Twisted-Python mailing list > > > [email protected] > > > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > > > _______________________________________________ > Twisted-Python mailing list > [email protected] > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > --000000000000034e110596baf405 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">If you need guaranteed=C2=A0delivery of t= he data, why not just use a TCP connection to the unix socket, instead of a= UDP connection which inherently can lose data?=C2=A0 In that case I don= 9;t think your patch would be needed.<input name=3D"virtru-metadata" type= =3D"hidden" value=3D"{"email-policy":{"state":"clo= sed","expirationUnit":"days","disableCopyPast= e":false,"disablePrint":false,"disableForwarding":= false,"enableNoauth":false,"persistentProtection":false= ,"expandedWatermarking":false,"expires":false,"isM= anaged":false},"attachments":{},"compose-id":"= ;1","compose-window":{"secure":false}}"><div><br><= /div><div>I didn't look at the source, so perhaps I missed something.</= div></div><br><div class=3D"gmail_quote" style=3D""><div dir=3D"ltr" class= =3D"gmail_attr">On Wed, Nov 6, 2019 at 9:10 AM Scott, Barry <<a href=3D"= mailto:[email protected]">[email protected]</a>> wrote= :<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.= 8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wednesday, = 6 November 2019 16:43:52 GMT Waqar Khan wrote:<br> > Hi Barry,<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Thanks for the response. Where can I = read more about (1). It seems<br> > like that is something I need to explore.<br> > As we already have (2) (cache for each process).<br> > Thanks again for your help.<br> <br> We use the UDS (Unix domain sockets) to talk to a master process.<br> Twisted has support for this. But you need a small patch to avoid data lose= .<br> <br> UDS does not lose data and is message based, not bytes based. We<br> use pickle to encode requests and responses.<br> <br> Barry<br> <br> The patch is:<br> <br> --- Twisted-18.4.0.orig/src/twisted/internet/unix.py.orig=C2=A0 =C2=A0 =C2= =A0 =C2=A02018-08-01 <br> 12:45:38.711115425 +0100<br> +++ Twisted-18.4.0/src/twisted/internet/unix.py 2018-08-01 12:45:47.9461151= 23 <br> +0100<br> @@ -509,11 +509,6 @@<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return self.w= rite(datagram, address)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0elif no =3D=3D EMSGSIZE:<br= > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise error.M= essageLengthError("message too long")<br> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 elif no =3D=3D EAGAIN:<br> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # oh, well, drop t= he data. The only difference from UDP<br> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # is that UDP won&= #39;t ever notice.<br> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # TODO: add TCP-li= ke buffering<br> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pass<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0else:<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise<br> <br> You then have to handle the EAGAIN error and do retries yourself.<br> As it stands the patch is not good enough to put into twisted as a<br> full fix would need to put the handling of the retries into twisted.<br> <br> I guess (2) does not work for you as the cache hit rate is low<br> and you need to share the cache to get a benefit. Cache entries<br> only get used a few times?<br> <br> In our case the hit rate is high (99%+) and we just pay the cost of<br> populating the caches on process start up, which ends up being<br> noise.<br> <br> Barry<br> <br> > <br> > On Wed, Nov 6, 2019 at 8:39 AM Scott, Barry <<a href=3D"mailto:barr= [email protected]" target=3D"_blank">[email protected]</a>>= ;<br> > <br> > wrote:<br> > > On Wednesday, 6 November 2019 14:21:22 GMT Maarten ter Huurne wro= te:<br> > > > On Wednesday, 6 November 2019 07:19:56 CET Waqar Khan wrote:= <br> > > > > Hi,<br> > > > > So, I am writing a twisted server. This server spawn mu= ltiple child<br> > > > > processes using reactor spawnProcess that initializes a= process<br> > > > > protocol.<br> > > > > <br> > > > > Now, each of the childprocess receives some REST reques= ts. Each<br> > > > > process has a dict that acts as cache.<br> > > > > Now, I want to share dict across processes.<br> > > > > In general, python has SharedMemoryManager in multiproc= essing module<br> > > > > which would have helped.<br> > > > > <a href=3D"https://docs.python.org/3/library/multiproce= ssing.shared_memory.html#m" rel=3D"noreferrer" target=3D"_blank">https://do= cs.python.org/3/library/multiprocessing.shared_memory.html#m</a><br> > > > > ultiprocessing.managers.SharedMemoryManager.SharedMemor= y But since I<br> > > > > am using twisted internal process implementation, how d= o I share this<br> > > > > dict across the processes so that all the processes use= this common<br> > > > > cache?<br> > > > <br> > > > Keeping a dictionary in SharedMemoryManager seems far from t= rivial. I<br> > > > don't think you can allocate arbitrary Python objects in= the shared<br> > > > memory and even if you could, you would run into problems wh= en one<br> > > > process mutates the dictionary while another is looking up s= omething or<br> > > > also mutating it.<br> > > > <br> > > > It could in theory work if you implement a custom lock-less = dictionary,<br> > > > but that would be a lot of work and hard to get right. Also = having<br> > > > shared memory mutations be synced between multiple CPU cores= could<br> > > > degrade performance, since keeping core-local CPU caches in = sync is<br> > > > expensive.<br> > > > <br> > > > Would it be an option to have only one process accept the RE= ST requests,<br> > > > check whether the result is in the cache and only distribute= work to the<br> > > > other processes if you get a cache miss? Typically the case = where an<br> > > > answer is cached is pretty fast, so perhaps you don't ne= ed multiple<br> > > > processes to handle incoming requests.<br> > > <br> > > We have used a couple of ways to cache.<br> > > 1. Use a singleton process to hold the cache and ask it, via IPC,= for<br> > > answers<br> > > from the other process.<br> > > 2. have a cache in each process<br> > > <br> > > Barry<br> > > <br> > > > Bye,<br> > > > <br> > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Maarte= n<br> > > > <br> > > > _______________________________________________<br> > > > Twisted-Python mailing list<br> > > > <a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a><br> > > > <a href=3D"https://twistedmatrix.com/cgi-bin/mailman/listinf= o/twisted-python" rel=3D"noreferrer" target=3D"_blank">https://twistedmatri= x.com/cgi-bin/mailman/listinfo/twisted-python</a><br> > > <br> > > _______________________________________________<br> > > Twisted-Python mailing list<br> > > <a href=3D"mailto:[email protected]" target=3D"_bl= ank">[email protected]</a><br> > > <a href=3D"https://twistedmatrix.com/cgi-bin/mailman/listinfo/twi= sted-python" rel=3D"noreferrer" target=3D"_blank">https://twistedmatrix.com= /cgi-bin/mailman/listinfo/twisted-python</a><br> <br> <br> <br> <br> _______________________________________________<br> Twisted-Python mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Twist= [email protected]</a><br> <a href=3D"https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-pytho= n" rel=3D"noreferrer" target=3D"_blank">https://twistedmatrix.com/cgi-bin/m= ailman/listinfo/twisted-python</a><br> </blockquote></div></div> --000000000000034e110596baf405-- --===============3567177367930515327== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVHdpc3RlZC1Q eXRob24gbWFpbGluZyBsaXN0ClR3aXN0ZWQtUHl0aG9uQHR3aXN0ZWRtYXRyaXguY29tCmh0dHBz Oi8vdHdpc3RlZG1hdHJpeC5jb20vY2dpLWJpbi9tYWlsbWFuL2xpc3RpbmZvL3R3aXN0ZWQtcHl0 aG9uCg== --===============3567177367930515327==--