Zeo and two processes changing the zodb - not getting the up to date version of the object

I have posted this on substnced as well.

So I am using celery to maintain a solr index of my zodb database. I am 
using substanced cms and zeo. 

Celery is running under supervisor and I am using deferred indexing as 
described in the substanced documentation.

Once I set celery going the first batch of tasks will execute beautifully 
but any of the next batch will say they are successful but don't show any 
of the changes I have made in zodb.  I had assumed this was something to do 
with my celery setup but I think it may be zeo returning the old unchanged 
object and not the new one.

I read this post and it seems very similar.

https://groups.google.com/forum/#!searchin/zodb/zeo|sort:relevance/zodb/wJ34URX5zp0/ZXODMcEvEwAJ


substanced object changed - event modified triggered which has two 
subscribers.
1 - index catalog of substanced

2- make updates to solr

Process 1 is run via a script sd_drain_indexing controlled by supervisor

Process 2 is run using celery which is passed the UUID of the substanced 
object that has changed.

To allow the indexing to happen in 1 I am delaying 2 for two minutes.

However 2 doesn't seem to pick up the latest version of the object. If I 
restart celery then I get the latest version but otherwise I am getting a 
'stale' version of the object.

How can I ensure that I am getting a current version of the object?  I am 
using the following code in the __call__ method of my basetask.



class TalbotTask(BaseTask):
    """Abstract base class for all tasks in my app."""
    #

    def __call__(self, *args, **kwargs):
        if getattr(self, "registry", None) is None:
            registry = App.conf['PYRAMID_REGISTRY']
            env = bootstrap(registry.settings[u'ini_file_celery'])
            root = env['root']
            self.registry = registry
            self.root = root
            self.solrFeed = TalbotSolrFeeder(self.registry, TALBOT_LOG)
            self.objectmap = find_objectmap(self.root)
        super(BaseTask, self).__call__(*args, **kwargs)



I then tried adding in an after commit hook but think this isn't relevant 
as the commit I want to pick up is the one from sd_drain_indexing.

I also took out the if statement in the above code so I always recreated 
the root. This worked initially but falls over because I have 'too many 
open files'. 

Any help?

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.