zodbconvert to Oracle very slow

Nicolas Göddel <[email protected]> Thu, 13 Sep 2018 07:52:33 -0700 (PDT)
Newsgroups gmane.comp.web.zope.zodb
Message-ID <[email protected]>
Hi,

we are developing a project for a customer using Plone 5.0.7 (later 5.1) 
and RelStorage 1.6.3. For developing we are using our own instance of 
Oracle 12.1.0.2.0 in our own infrastructure. It works good and fast and 
without any issues.

Now our customer needs our application as a docker image which will be 
build from our source within in their infrastructure. Because there are 
some settings and predefined objects in Plone we created on our side and 
which have to always be there when the docker image is started, I always do 
a zodbpack on our database and then a zodbconvert from our RelStorage to a 
Data.fs file and blobstorage directory. This process lasts nearly 10 
minutes and produces a Data.fs of 279 MB and a blobstorage of 1,3 MB. Then 
I tar.gz them to a size of 46 MB and send it over to our customers docker 
build platform. The database will then be integrated in the docker image 
and zodbconvert will be called at the start of the docker application 
before Plone if and only if the Oracle database is empty or if it is a new 
version of the initial database. But that is the culprit. Converting the 
Data.fs and blobstorage back to our customers Oracle instance lasts over 7 
hours and in the end it runs with 0.008 MB/s. But as soon as the conversion 
is done Plone itself gets started and runs very smoothly, maybe a little 
bit slower than on our machines, but with a bigger object cache it is very 
good.
So the main thing for me is the slowness of zodbconvert. Is there something 
what I can do to speed it up? Or do you have an other idea instead of 
zodbconvert to fill the database quickly with the content of our database? 
Sometimes there are tids with more than 50,000 records, e.g. one had 86457 
records and it lasts nearly 90 minutes until the next tid was processed.

What else do you need to know about my scenario? What information can I 
give you to track down this slowness issue?

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.