RE: Persistent storage / forwarding of events

Oh, we like Ruby.  Greg Burd wrote a Ruby pubsub client library
which we distribute with mod_pubsub.

Two possible solutions to the O(n^2) problem...

1. Perhaps one solution to the explosion would be to remove events
from a topic once you've consumed them?

2. Another solution would be to keep the timestamp of the last received
event, add a fudge factor of extra time to that, and use that as the
do_max_age of the subscription.  Reason you need a fudge factor is
because time-based replay is against the event publication time,
and there is some latency in delivering events.

Do either of those work for you?

   Adam


-----Original Message-----
From: Jens Alfke [mailto:[email protected]]
Sent: Thursday, October 16, 2003 4:33 PM
To: Adam Rifkin
Cc: [email protected]; Ben Sittler
Subject: Re: [Mod-pubsub-developer] Persistent storage / forwarding of
events

On Oct 16, 2003, at 4:04 PM, Adam Rifkin wrote:

> I'll take a moment to put in a plug for Python.  Most of the pubsub
> apps I write that don't use JavaScript use Python.

I've been getting into Ruby lately, which is very nice, and I don't 
want to confuse myself by learning too many languages at the same time 
:)

> So we'd write code to do this as part of the app that is able to squash
> duplicates received before the last connection dropped.  Note that
> do_max_age of infinity on the subscription lets a client replay all
> the events in the topic so the app should be able to ascertain where
> it last left off.

Hm. This seems inefficient — I've run into the same issue with the 
Blogger API. I have to do something like:
	"OK server, give me the last 10 events."
	(Hmm, I don't recognize any of those, better ask for more...)
	"OK server, give me the last _20_ events."
	(Ah, I already had #16-20, so now I'm caught up.)
So the server had to send me 30 events (10 of them duplicates!) just so 
I could get the 15 new ones. And this is an O(n^2) algorithm, so it 
gets even worse if there are even more messages to sync up.

--Jens


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php