Re: ZEO as a kind of memcache

> On Aug 7, 2017, at 2:44 PM, Rafael Oliveira <[email protected]> wrote:
> 
> It seems like a textbook case for memcache. However, I already have ZEO, which is already there, talking to all instances. So, why not use ZEO as a replacement for memcache?

Because you want to be very careful about not inducing a write on otherwise "read" transactions.  This can greatly increase chance for conflicts, and other sorts of problems.  

IMHO, the cases where it might be reasonable to use ZODB to persistently store your summarized data/calculation/rollup:

(1) On transactions that are otherwise writing (e.g. not a GET request to a web application), such as a save operation.

(2) On read-oriented transactions for which you queue the result into a job queue using Redis or similar.

(3) Only use ZODB if your calculated result is and should be durable across transactions (e.g. there is justification to persist, not just to cache).


In my own data-analysis (time-series reporting) applications we store summarized data in a "cache" (OOBTree-based) but only carefully modify this cache to be keyed with tuples of UUID and modification timestamp of original data sources, and only ever on already-writing transactions (e.g. on zope.lifecycleevent.interfaces.IObjectModifiedEvent subscriber).  You have to take good care to get this right.

If you just want to cache, use a cache.  If you want to and need to persist, then do it on save transactions or via a job queue.

/...My $0.02...

Sean

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.