Re: Upgraded Pyro3->Pyro4. Seem to have a Proxy deadlock issue.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 28-4-2012 22:11, Isaac Jurado wrote:
> On Sat, Apr 28, 2012 at 7:24 PM, Todd Williamson <[email protected]> wrote:
>> 
>> Over the last year or so I've developed a new Python distributed programming
>> package that uses Pyro to provide a lot of really cool concurrency mechanisms.
>> Up to this point, I've only used Pyro3, but before I release the package, I
>> wanted to see if I could get things working with Pyro4.  I was surprised how few
>> changes I really had to make to get things running.  It was almost just a global
>> replace for ensuring a nameserver is running on the local subnet, 
>> Pyro.core.getProxyForURI-> Pyro4.Proxy, and manually registering the uri object
>> returned from the Pyro Daemon with the nameserver.  With that said, I tried
>> running my verification suite and ended up with mixed results.  Sometimes my
>> programs work completely and sometimes they would deadlock.
> 
> At work, we ran through the exact same story: had proper working code based on
> Pyro3 and after upgrading to Pyro4 we started suffering random deadlocks (or, at
> least, a blocked server) so we had to revert back to Pyro3.
> 
> Also, we were unable to find an explanation; although we couldn't dedicate as much
> time to find it out.
> 
>> I have noticed that I can open up a Python interpreter, when the program hangs,
>> and manually create a new proxy for the same receiver object that seems to be
>> causing the hanging.  Subsequently, if I make any method call to this new proxy,
>> it unhangs the program above.  The program then runs to completion.  Really weird
>> stuff.
> 
> This is something we didn't find out.  Which, as you say, makes the situation even
> more puzzling.
> 
> I'm quite intrigued to see if a solution/explanation comes up.
> 
> Best wishes.
> 

Todd, Isaac: someone has contacted me privately and reported a possible error in
Pyro4's threadpool implementation that could cause undesired blocking behavior.
I haven't had the time to sort this out yet, but in the meantime you could try one of
these to see if the problem disappears:

- - set the threadpool size to a fixed number of threads  (set THREADPOOL_MINTHREADS and
THREADPOOL_MAXTHREADS to the same value)
- - switch to the select/poll based multiplex server (SERVERTYPE=multiplex)

Could you try each of these and let us know if the problem disappears?
It can help me diagnose the cause of the problem.


Regards
Irmen de Jong

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPnGkkAAoJEHC9o7RHFlf27iMIALE/eMNuNxiJcmeg6oizUdO3
39+QY04WQ8zUV38Gn8YeFSoZ/khbUyYRk2vDrfX69DXGAiANrFQgGMu6T6yxVo7Y
Qq7lQof85TBos4x5hRKRLF24aJGsZ/RD1fM4bdFGj+YYoxr9oo23tXRLIMJsR7Wz
/xvK+8V9GxLL7x5z+b+uM0rhgyJ/OvC5Y935chV1Cdo6II74kxfIBvtHaM91Fv+0
CSGs0+srI09mDfhClp+sIrAbDT3tCm3tndEaTvcTjNMaSA8ZE466aUvi+gQrI2DU
KnUZ+vlS10OzMeR5+7xVl7G7PL922p+l1eyiWFiVe3AAB3Nw6EGgxrTX71kQ6MU=
=0QCj
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/