zodbconvert to Oracle very slow
Nicolas Göddel <[email protected]> Thu, 13 Sep 2018 07:52:33 -0700 (PDT)
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
Hi, we are developing a project for a customer using Plone 5.0.7 (later 5.1) and RelStorage 1.6.3. For developing we are using our own instance of Oracle 12.1.0.2.0 in our own infrastructure. It works good and fast and without any issues. Now our customer needs our application as a docker image which will be build from our source within in their infrastructure. Because there are some settings and predefined objects in Plone we created on our side and which have to always be there when the docker image is started, I always do a zodbpack on our database and then a zodbconvert from our RelStorage to a Data.fs file and blobstorage directory. This process lasts nearly 10 minutes and produces a Data.fs of 279 MB and a blobstorage of 1,3 MB. Then I tar.gz them to a size of 46 MB and send it over to our customers docker build platform. The database will then be integrated in the docker image and zodbconvert will be called at the start of the docker application before Plone if and only if the Oracle database is empty or if it is a new version of the initial database. But that is the culprit. Converting the Data.fs and blobstorage back to our customers Oracle instance lasts over 7 hours and in the end it runs with 0.008 MB/s. But as soon as the conversion is done Plone itself gets started and runs very smoothly, maybe a little bit slower than on our machines, but with a bigger object cache it is very good. So the main thing for me is the slowness of zodbconvert. Is there something what I can do to speed it up? Or do you have an other idea instead of zodbconvert to fill the database quickly with the content of our database? Sometimes there are tids with more than 50,000 records, e.g. one had 86457 records and it lasts nearly 90 minutes until the next tid was processed. What else do you need to know about my scenario? What information can I give you to track down this slowness issue? -- You received this message because you are subscribed to the Google Groups "zodb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.