Re: ZEO as a kind of memcache

On Mon, Aug 7, 2017 at 4:44 PM, Rafael Oliveira <[email protected]> wrote:

> Hi all,
>
> I have a Zope 2 application, with multiple instances connecting to ZEO. I
> have expensive computation output that I want to cache, and I want to share
> this cache among the multiple instances I have.
>
> It seems like a textbook case for memcache. However, I already have ZEO,
> which is already there, talking to all instances. So, why not use ZEO as a
> replacement for memcache?
>

...


> It kind of works. I haven't tested it much yet. A problem I already found
> is that when I restart the ZEO server the Zope instances complain with this
> message: ClientStorageError: [('127.0.0.1', 3100)] Client has seen newer
> transactions than server!
>

This is because the client has a replica of a database that doesn't exist
anymore.

If you truly want this to be memory only, I think you'd want to find a way
to signal to the clients that they should all discard their caches.  Or, as
Sean suggested, store your results persistently.

Hm, see below.



>
> I'd like your feedback:
>
> - Is this a very crazy idea? Why?
>

I don't think it's a crazy.


> - What are the cons, comparing to memcache?
>

IDK but a potential major benefit is that it's easy to keep the cache
current, because it's essentially a replica.


> - Should I use other storage type, like mappingstorage?
>

IDK if anyone is maintaining tempstorage any more.  But you may want to
have something more persistent than an in-memory storage.

You might be able to get around the client cache issue by creating a
slightly customized mapping storage that initializes itself with a new
transaction on startup.  There's a dance between the clients, server, and
storage such that when you restarted the server, the clients would discard
their caches.

See
https://github.com/zopefoundation/ZEO/blob/master/src/ZEO/interfaces.py#L101
and
https://github.com/zopefoundation/ZEO/blob/master/src/ZEO/StorageServer.py#L769

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.