Re: Application change notification

On Thu, Mar 16, 2017 at 12:00 PM, Jason Madden <[email protected]
> wrote:

>
> > On Mar 16, 2017, at 10:08, Jim Fulton <[email protected]> wrote:
> >
> >> Fundamentally, with the RelStorage implementation being delayed, the
> "real-time" aspect goes missing, which makes this a bit self-defeating :)
>
> > Well, except that this is fixable. RelStorage could support this using
> triggers, if it wanted to.   That's basically the approach that Newt will
> use
>
> That may work for PostgreSQL (LISTEN), but I don't know of any non-polling
> way to do that in stock MySQL. Oracle does have a publish/subscribe
> implementation, but last time I used it it was...flaky at best (and I'm not
> sure how it's exposed to Python, it may require polling too).
>

I don't think it's a good idea for RelStorage to be defined by the minimum
capabilities of its supported databases.


>
>
> > Even when data are spread over multiple objects, it's still usually easy
> enough to have an single object that's modified on changes. Consider
> watching changes to a big BTree.  You could watch an associated Length
> object, rather than watching changes to individual nodes.
>
> I think we're looking at a chicken-and-egg problem here. A Length object
> doesn't know its associated BTree in the zope.container.btree case, which
> is surely the most common case. If we knew ahead of time that we would want
> to know that a particular container was going to be modified, we might have
> set a __parent__ pointer, but we didn't and so we can't. I suppose that
> every time we decide we want to notice that a particular type of object is
> being modified we could crawl through the database and write __parent__
> pointers, but that doesn't seem scalable. Or in this particular case we
> could write code like:
>
>    container[item] = value
>    try:
>      # make sure there's a back pointer
>      container.__BTreeContainer_len.__parent__ = container
>    except AttributeError:
>      pass
>
> But that's ugly. And we may not have control of the code that sets items
> anyway.
>
> I suppose one could also argue that each and every time you want to start
> noticing that some object has changed, you register a new subscriber:
>
>   container = get_the_container_for_editing()
>
>   len_oid = container.__BTreeContainer.len._p_oid
>   def callback(oids):
>     if len_oid in oids:
>        tell_somebody_the_collection_changed(container)
>
>   db.subscribe(callback)
>
> But that breaks as soon as you have more than one process involved
> handling requests (unless you invent another mechanism to centralize such
> subscribers, but then we're back to a different kind of chicken-and-egg
> problem).
>

When you're designing a feature for an application, that's often going to
be reflected in the schema.  If you add a new feature, that may require a
change to the schema. That's why we have tools like zope.generations.

There's nothing magic here.


>
> > Certainly, some application machinery is needed to leverage this.
> Having Redis running on your network didn't magically give your application
> a real-time UI. It provided something you could build on.  The same applies
> here.
>
> That's all very true. It all takes some thought.
>
> I suppose the point I'm trying to make is that it looks like the machinery
> needed to leverage an OID-based system is going to be more complicated,
> more finicky, and only a partial solution anyway


Well, an earlier versions of this worked very well for the the applications
I tried it with. But both of those applications used generational sets,
which were designed to be used with this mechanism (and for specific
application requirements).



> ---had it been available, it seems likely we would have still gone with a
> zope.event+Redis based system. It seems better to have an extra tool that's
> good at what it's doing than to try to compromise with a less flexible tool
> just because it's already there.
>

YMMV


>
> Now, even though our application is heavily based on the usual Zope 3
> paradigms, it's possible its needs are outside the mainstream. And it's
> also possible I'm looking at this through too-heavily Redis colored
> glasses. So I'm *not* trying to say this has no use and can't ever work for
> any application, I'm only sharing my thoughts based on my own experience,
> for whatever that's worth. That /was/ the original question :)
>

Fair enough.

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.