Re: experimental ZODB prefetch support in ZEO
Julien Muchembled <[email protected]>
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
Le 07/15/16 à 14:47, Jim Fulton a écrit : > For years, people have been asking for a bulk load feature, especially > in ZEO and especially to aid startup. For years, I've been threatening > to add a prefetch feature to address this requirement. I finally took > the first step towards this yesterday. Prefetching is also something we'd like to do at Nexedi, because it would certainly speed up our applications, with ZEO or NEO, not only at startup. However, the biggest issue for us is to know which oids to prefetch. In fact, it's so difficult that we though about something different than an explicit method call to the connection or storage. In ERP5, requests that loads many oids and for which we have an idea to optimize are those that loop over several objects, like children of the same folder. We think that with 'greenlet', we could manage to process iterations of such loops in parallel: whenever a load is about to wait for IO, the current iteration would be suspended and another would start/resume. This can lead to several tasklets loading oids at the same time, which is also a good thing because only one network packet could be sent to retrieve these oids. More generally, we're more interested in something that would allow the caller to continue execution elsewhere whenever the storage has to wait. Julien -- You received this message because you are subscribed to the Google Groups "zodb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.