Re: Zeo and two processes changing the zodb - not getting the up to date version of the object

On Wed, Feb 1, 2017 at 10:02 AM, Jo Gilder <[email protected]>
wrote:

> I have posted this on substnced as well.
>
> So I am using celery to maintain a solr index of my zodb database. I am
> using substanced cms and zeo.
>
> Celery is running under supervisor and I am using deferred indexing as
> described in the substanced documentation.
>
> Once I set celery going the first batch of tasks will execute beautifully
> but any of the next batch will say they are successful but don't show any
> of the changes I have made in zodb.  I had assumed this was something to do
> with my celery setup but I think it may be zeo returning the old unchanged
> object and not the new one.
>
> I read this post and it seems very similar.
>
> https://groups.google.com/forum/#!searchin/zodb/zeo|sort:
> relevance/zodb/wJ34URX5zp0/ZXODMcEvEwAJ
>
>
> substanced object changed - event modified triggered which has two
> subscribers.
> 1 - index catalog of substanced
>
> 2- make updates to solr
>
> Process 1 is run via a script sd_drain_indexing controlled by supervisor
>
> Process 2 is run using celery which is passed the UUID of the substanced
> object that has changed.
>
> To allow the indexing to happen in 1 I am delaying 2 for two minutes.
>
> However 2 doesn't seem to pick up the latest version of the object. If I
> restart celery then I get the latest version but otherwise I am getting a
> 'stale' version of the object.
>
> How can I ensure that I am getting a current version of the object?  I am
> using the following code in the __call__ method of my basetask.
>
>
>
> class TalbotTask(BaseTask):
>     """Abstract base class for all tasks in my app."""
>     #
>
>     def __call__(self, *args, **kwargs):
>         if getattr(self, "registry", None) is None:
>             registry = App.conf['PYRAMID_REGISTRY']
>             env = bootstrap(registry.settings[u'ini_file_celery'])
>             root = env['root']
>             self.registry = registry
>             self.root = root
>             self.solrFeed = TalbotSolrFeeder(self.registry, TALBOT_LOG)
>             self.objectmap = find_objectmap(self.root)
>         super(BaseTask, self).__call__(*args, **kwargs)
>
>
>
> I then tried adding in an after commit hook but think this isn't relevant
> as the commit I want to pick up is the one from sd_drain_indexing.
>
> I also took out the if statement in the above code so I always recreated
> the root. This worked initially but falls over because I have 'too many
> open files'.
>
> Any help?
>

Assuming that you're running ZEO 5, and that you're using `<clientstorage>
configuration element, I suggest adding the option:

  server-sync true

To your clientstorage configuration.  This forces a client to process any
updates that were in-flight before starting a transaction.

If you aren't using ZEO 5 or the new <clientstorage> configuration tag,
then you'll need to upgrade and update your configuration.

See: https://github.com/zopefoundation/ZEO#configuration-strings-files

Jim


-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.