Re: Pyro 3 event server hang

Dear Irmen,

thanks for your message. We started using Pyro3 and it's event service when it already was in maintenance. Our hope was that it is a stable piece that we can build our project on. To a large part, this worked out nicely.

On 21 Apr 2013, at 15:27, Irmen de Jong <[email protected]> wrote:

> On 21-4-2013 12:05, Christoph Gohle wrote:
>> Hi,
>> 
>> we are running pyro 3 here and are observing a strange behaviour of it's event service.
>> After a certain sequence of events (which is not quite clear yet, but we have a script
>> that reproduces the behaviour) (one of the) worker threads blocks and stops delivering
>> events. I think I have seen it waiting for the self.remote.lock in
>> self.remote.remoteInvocation, which indicates that a previous remoteInvocation never
>> returned, which if I read the code correctly, should never happen.
>> 
>> Is this something that people have seen before? Is there a reason know and if so is
>> there a fix? If not, can I get some hint, what to try?
> 
> But have you tried playing with the PYRO_ES_QUEUESIZE or PYRO_ES_BLOCKQUEUE config
> items? Or maybe you're overloading the system and you need to throttle your event
> producers a little?
Event rate is definitely not too high. The queue only starts filling after the worker has stopped delivering. I did play with the PYRO_ES_BLOCKQUEUE. That helps keeping the producers happy (as they don't block once the worker is and it's queue fills). But of course this does not fix that the listener does not recieve events anymore. I checked that the event handler finishes normally after the last event it recieved. I found that we are running an older version of Pyro on our event server machine (3.14) and I have seen that some modifications regarding timeouts were done in 3.16. Could it be that the socket.send is blocking in the worker? I will try to upgrade and see if it helps.
> 
> Pyro3 is deprecated anyway. The last version is over a year old. Can't you upgrade to
> Pyro4? (Yeah, you'll have to implement an event service yourself, because Pyro4 doesn't
> supply an out of the box solution for this).
Yes, this might be an option. Although we were hoping to not have to do the debugging of an event service our selves… oh well.
> 
> I haven't touched Pyro3 for over a year and won't be getting back to it. The website and
> doc say that Pyro3 is "in maintenance mode" but I'll change this now to "no longer
> maintained". Unless it is a trivial bugfix, but debugging race conditions in the event
> server isn't one of these, sorry.
Yes, debugging race conditions is a pain. That's why we liked the idea of using established code. 

Anyway, I thank you very much for this nice package. Apart from this issue we are very happy with it's features and ease of use.

Best,
Christoph


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter