Re: Application change notification

> On Mar 16, 2017, at 10:08, Jim Fulton <[email protected]> wrote:
> 
>> Fundamentally, with the RelStorage implementation being delayed, the "real-time" aspect goes missing, which makes this a bit self-defeating :)

> Well, except that this is fixable. RelStorage could support this using triggers, if it wanted to.   That's basically the approach that Newt will use

That may work for PostgreSQL (LISTEN), but I don't know of any non-polling way to do that in stock MySQL. Oracle does have a publish/subscribe implementation, but last time I used it it was...flaky at best (and I'm not sure how it's exposed to Python, it may require polling too). 


> Even when data are spread over multiple objects, it's still usually easy enough to have an single object that's modified on changes. Consider watching changes to a big BTree.  You could watch an associated Length object, rather than watching changes to individual nodes.

I think we're looking at a chicken-and-egg problem here. A Length object doesn't know its associated BTree in the zope.container.btree case, which is surely the most common case. If we knew ahead of time that we would want to know that a particular container was going to be modified, we might have set a __parent__ pointer, but we didn't and so we can't. I suppose that every time we decide we want to notice that a particular type of object is being modified we could crawl through the database and write __parent__ pointers, but that doesn't seem scalable. Or in this particular case we could write code like:

   container[item] = value
   try:
     # make sure there's a back pointer
     container.__BTreeContainer_len.__parent__ = container
   except AttributeError:
     pass

But that's ugly. And we may not have control of the code that sets items anyway.

I suppose one could also argue that each and every time you want to start noticing that some object has changed, you register a new subscriber:

  container = get_the_container_for_editing()
  
  len_oid = container.__BTreeContainer.len._p_oid
  def callback(oids):
    if len_oid in oids:
       tell_somebody_the_collection_changed(container)

  db.subscribe(callback)

But that breaks as soon as you have more than one process involved handling requests (unless you invent another mechanism to centralize such subscribers, but then we're back to a different kind of chicken-and-egg problem).

> Certainly, some application machinery is needed to leverage this.  Having Redis running on your network didn't magically give your application a real-time UI. It provided something you could build on.  The same applies here.

That's all very true. It all takes some thought. 

I suppose the point I'm trying to make is that it looks like the machinery needed to leverage an OID-based system is going to be more complicated, more finicky, and only a partial solution anyway---had it been available, it seems likely we would have still gone with a zope.event+Redis based system. It seems better to have an extra tool that's good at what it's doing than to try to compromise with a less flexible tool just because it's already there.

Now, even though our application is heavily based on the usual Zope 3 paradigms, it's possible its needs are outside the mainstream. And it's also possible I'm looking at this through too-heavily Redis colored glasses. So I'm *not* trying to say this has no use and can't ever work for any application, I'm only sharing my thoughts based on my own experience, for whatever that's worth. That /was/ the original question :)

Jason


-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.