Re: Can I export all the data (incl refs between objects) and reimport it into a new zodb instance

On Tue, Oct 4, 2016 at 10:18 AM, Jo Gilder <[email protected]> wrote:
> So,
>
> I have two sites using substanced. One has data I imported from a csv file,
> the other has data that was imported from postgres. There are several
> references between the objects, these are based on UUID aren't they?

No, they are sequentially-assigned ids (simpler to RDBMS serials).

Are these 2 sites in the same database?


>
> In both cases the data is now 'live' and people are editing it.
>
> Two things:
>
> I want to have an exportable representation of the important data - ie a
> backup that isn't a data.fs file.

Why is not being a data.fs file a requirement?


> This data could store the UUID of each
> record. substanced does have something called sd-dump which stores
> everything in yaml files but I have lots of records and it falls over trying
> to create everything.

There's a database export feature that I wrote a very long time ago.
I haven't used it in many years.  It's exposed by Zope. I suggest you
take at the ZODB ExportImport module.  It would be starting at some
object and exporting all objects reachable from that object. It
doesn't handle cycles though.


>
> We have also been asked to add to the schema for these objects - to add in
> extra fields.

This is usually easy. See:

http://www.zodb.org/en/latest/guide/writing-persistent-objects.html#schema-migration

>
> Now if they hadn't changed any of the data I would just reimport everything
> from postgres or csv and recreate all the links but as I don't want to lose
> anything they have changed I can't do that.
>
> Is there a way to export the objects and all the references and then
> reimport into a new data.fs along with the additional data?

Theoretically, but I wouldn't recommend it.

I suggest adding the field with a default with a class attribute and
then writing a script to update the existing records with the new
data.


> Can you force a
> new object to have a particular UUID so I can maintain the relationships?

Not with a public API. Why not just update the existing data?

> I don't have a record of the UUIDs of each of the objects that links it to
> the original data in the csv file or the postgres database. I do have
> records of the 'names' (one of the properties) of the new objects and the
> uniqueID in the postgres database tables

So assuming you can find the records in the data that needed to be
updated, you could use the name to get the data needed to update the
objects.

Jim


-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.