Re: Non-blocking publishing in Python
[email protected] Wed, 6 Apr 2005 06:20:01 -0500
| Newsgroups | gmane.comp.web.mod-pubsub.devel |
|---|---|
| Message-ID | <[email protected]> |
Adam: My basic criteria was finding a message-based system that was being activ= ely developed/supported with Python bindings. I looked at a number of open so= urce JMS implementations, most of which had no Python bindings. Activ MQ and M= om4J were the two that had support for Python, but Activ MQ seems too Windows-centric for serious use on Linux. Mom4J looked more promising, bu= t their implementation was incomplete and their user community was like a g= host town. There's really no point in investing time in a dead project (unless= it has killer technology). When I discovered mod_pubsub and the REST-based s= ystems (been on my radar for a long time, but never delved into it) I was happy = to find out that they overcome the confinement limitations of JMS. At this p= oint, I'm ignorant of the security ramifications that this presents, which I'd = like to find out more. FWIW, Mom4J provided hooks for authentication, but no a= ctual implementation was present. I'll probably be looking at some other REST systems, and I'll let you kno= w the outcome. Brian Quoting [email protected]: > In the interest of strengthening everyone's knowledge base, what did yo= u=20 > compare against and what were your concerns? I've noticed that=20 > xmlBlaster and Activ MQ both offer HTTP oriented methodologies for=20 > asynchronous notification. But I am not sure how robust they are, nor=20 > how native/central to the design they are, nor how well they work in=20 > javascript contexts. >=20 > A. >=20 > [email protected] wrote: >=20 > >Adam: > > > > =20 > > > >>The question was not bone-headed at all. It's just that sometimes our > >>Python guy goes into { while 1: sleep } mode. ;) > >> =20 > >> > > > >In that case, I'll continue to poll() the list. Hopefully the results = aren't > too > >asynchronous! > > > > > > =20 > > > >>Both of those client libraries were experimental prototypes, as is > >>probably clear from reading the source. Plus, I was kind of learning > >>Python as we wrote them. > >> > >>I'm Cc'ing Ben in case he knows the answer to your embedding question= . > >> =20 > >> > > > >So is anyone using the Python clients for any interesting applications= ? > I've > >evaluated some other asynchronous messaging technologies, and mod/re-p= ubsub > >looks the most promising w.r.t. Python and Internet-wide messaging. It= would > be > >great if anyone could share some insight into the embedding question, = or how > to > >harness the non-blocking behavior of the pubsublib. > > > > > > =20 > > > >>I do know that no one is currently working to port the client library > >>to Twisted; right now all the energy and effort is going into > >>rewriting the server using Twisted, and hopefully we can learn from > >>that to do a client rewrite sometime later. If you're feeling > >>ambitious, feel free to take a crack at doing it with Twisted's > >>Deferred mechanism... > >> =20 > >> > > > >How is the progress on the server side? Is the current version stable = enough > to > >be making changes that would be compatible with the current server > development? > >I have some experience with Twisted, so I wouldn't mind taking a look,= but > I > >would like to get a sense of the project road map so my work isn't was= ted. > > > > =20 > > > >> -- Adam > >> =20 > >> > > > >Brian > > > > > >PS Are you Adam Rifkin or another Adam? > > > > > > =20 > > > >>On Apr 5, 2005 4:13 AM, > >>[email protected] > >><[email protected]> wrote: > >> =20 > >> > >>>Haven't heard anything back from the list regarding this question. I= f it > >>> =20 > >>> > >>is > >> =20 > >> > >>>because the question was too bone-headed, then there is no remedy. > >>> =20 > >>> > >>Otherwise, > >> =20 > >> > >>>the following information might be useful to someone else getting > started. > >>> > >>>There are two python client libraries: pubsublib.py and libkn/libkn.= py. > >>>According to the source, pubsublib.py is a non-blocking implementati= on, > >>> =20 > >>> > >>while > >> =20 > >> > >>>libkn.py is blocking. The examples in apps are built off of pubsubli= b, so > >>> =20 > >>> > >>they > >> =20 > >> > >>>should be non-blocking. > >>> > >>>This seems to be true when running as a standalone script, but what = about > >>> =20 > >>> > >>when I > >> =20 > >> > >>>want to embed? Is the logging package being used to help the debuggi= ng > >>> =20 > >>> > >>process > >> =20 > >> > >>>or are people using pdb? > >>> > >>>Also, I read somewhere that the client is supposed to be using Twist= ed in > >>> =20 > >>> > >>the > >> =20 > >> > >>>near future. Is this still the case, and if so, what's the progress = on > it? > >>> > >>>Thanks, > >>>Brian > >>> > >>> > >>>Quoting [email protected]: > >>> > >>> =20 > >>> > >>>>Hello: > >>>> > >>>>I'm trying to use the apps/publish.py module as a starting point fo= r > >>>> =20 > >>>> > >>writing > >> =20 > >> > >>>>my > >>>>own simple publishing lib to work with repubsub. The script runs fi= ne on > >>>> =20 > >>>> > >>its > >> =20 > >> > >>>>own, but whenever I try to embed it in other code, I seem to get > >>>> =20 > >>>> > >>blocking > >> =20 > >> > >>>>issues and the CPU load spikes. > >>>> > >>>>Inside the apps/publish.py script, I looked at the following method > >>>> =20 > >>>> > >>calls: > >> =20 > >> > >>>> asyncore.poll() > >>>> scheduler.timeout() > >>>> scheduler.run() > >>>> > >>>>Now Python's asyncore does not have documentation on the poll funct= ion, > >>>>although > >>>>in my interpreter, I do see a reference for poll. The two scheduler > >>>> =20 > >>>> > >>function > >> =20 > >> > >>>>calls both pass in their implementations. If I comment out that loo= p, my > >>>>messages don't get published. So what's going on here? > >>>> > >>>>Ideally, I'd like to be able to use Twisted's Deferred mechanism to > >>>> =20 > >>>> > >>handle > >> =20 > >> > >>>>the > >>>>blocking issue, but I'm unsure the best way to approach it without > >>>> =20 > >>>> > >>heavily > >> =20 > >> > >>>>modifying the code base. > >>>> > >>>>Any help is greatly appreciated. > >>>> > >>>>Thanks, > >>>>Brian > >>>> > >>>>-- > >>>> =3D=3D=3D=3D > >>>>brian lee dae yung > >>>>biomimetic art and research > >>>> mux space . com > >>>> > >>>>---------------------------------------------------------------- > >>>>This message was sent using IMP, the Internet Messaging Program. > >>>> > >>>> > >>>> > >>>>------------------------------------------------------- > >>>>SF email is sponsored by - The IT Product Guide > >>>>Read honest & candid reviews on hundreds of IT Products from real u= sers. > >>>>Discover which products truly live up to the hype. Start reading no= w. > >>>>http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick > >>>>_______________________________________________ > >>>>Mod-pubsub-developer mailing list > >>>>[email protected] > >>>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer > >>>> > >>>> =20 > >>>> > >>>-- > >>> =3D=3D=3D=3D > >>>brian lee dae yung > >>>biomimetic art and research > >>> mux space . com > >>> > >>>---------------------------------------------------------------- > >>>This message was sent using IMP, the Internet Messaging Program. > >>> > >>>------------------------------------------------------- > >>>SF email is sponsored by - The IT Product Guide > >>>Read honest & candid reviews on hundreds of IT Products from real us= ers. > >>>Discover which products truly live up to the hype. Start reading now= . > >>>http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick > >>>_______________________________________________ > >>>Mod-pubsub-developer mailing list > >>>[email protected] > >>>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer > >>> > >>> =20 > >>> > >>------------------------------------------------------- > >>SF email is sponsored by - The IT Product Guide > >>Read honest & candid reviews on hundreds of IT Products from real use= rs. > >>Discover which products truly live up to the hype. Start reading now. > >>http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick > >>_______________________________________________ > >>Mod-pubsub-developer mailing list > >>[email protected] > >>https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer > >> > >> =20 > >> > > > > > > =20 > > >=20 >=20 >=20 > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users= . > Discover which products truly live up to the hype. Start reading now. > http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick > _______________________________________________ > Mod-pubsub-developer mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/mod-pubsub-developer >=20 --=20 =3D=3D=3D=3D brian lee dae yung biomimetic art and research mux space . com ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick