Re: Application change notification

On Thu, Mar 16, 2017 at 10:32 AM, Jason Madden <[email protected]
> wrote:

> Hi all,
>
> We maintain such a real-time ZODB/RelStorage-based web application so I
> can share my thoughts from that perspective.
>
> It's not clear to me that the method described here would be useful for us
> at an application level.


> The primary issue I see is that it's too low-level. Getting an OID of an
> object that's changed is not necessarily useful from an application
> perspective without further content. For example, a user may be editing a
> document that has an attachment that's a Blob. If the contents of the blob
> change, we'd get the OID notification for the Blob, but what we really want
> to show as changed---the containing document---didn't change from the ZODB
> perspective, so we wouldn't get notified about that.
>

It certainly *isn't sufficient*, at the application level, but it *enables*
higher-level mechanisms.

...

Fundamentally, with the RelStorage implementation being delayed, the
> "real-time" aspect goes missing, which makes this a bit self-defeating :)
>

Well, except that this is fixable. RelStorage could support this using
triggers, if it wanted to.   That's basically the approach that Newt will
use.  Implementing this over ZEO is, of course, trivial.


> So what do we do now when we want to deal with presenting users with
> changed information in real time? The application is more or less
> responsible for this: Using a system based on zope.event (and
> zope.lifecycleevent) the application can decide what changes are important
> to the users. Such important events are distributed across the server
> cluster in real-time using Redis.
>

That's cool, but I think there's value in having this without having to run
Redis.


> If a hypothetical ZODB-based change notification system were opt-in (such
> that the application specifies what OIDs it wants to have subscribers
> called for), that would solve part of the low-level problem---although
> still only for Persistent objects.


It's already opt-in. Nothing would happen if you didn't register a
subscriber.  I resisted subscribing on specific objects, because I viewed
that as only sugar to avoid the application having to filter oids, which is
easy enough. I realized though, in thinking about the Newt implementation,
that the storage could be more efficient if it knew what the application
was looking for, so I'm inclined to allow objects/oids to be passed to
subscribe.  In general however, with something like this, it's better to
start low-level and not add a lot of policy without application experience
using the low-level mechanism.

...

Certainly, some application machinery is needed to leverage this.  Having
Redis running on your network didn't magically give your application a
real-time UI. It provided something you could build on.  The same applies
here.

For example, for a two-tiered kanban project of mine, I use generational
sets, https://pypi.python.org/pypi/zc.generationalset.  The set captures
incremental changes providing a single objects I can watch for changes on.

It's actually easy for me to see how lots of model objects typically served
in a UI could work this way because most model objects are, in practice,
single persistent objects.

Even when data are spread over multiple objects, it's still usually easy
enough to have an single object that's modified on changes. Consider
watching changes to a big BTree.  You could watch an associated Length
object, rather than watching changes to individual nodes.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.