Re: Python-3-Migration of existing Data.fs

I like the high-level approach of analyzing objects at letting user provide
conversion rules in terms of application-level classes and attributes.
Thanks!

Some issues:

   - Conversion doesn't fix oids in persistent references. It looks like
   nobody has tried this with a non-toy dataset until now. :)

   - Conversion doesn't handle binary keys or values in mapping objects or
   items in sequences, or non-persistent subobjects.

   I suppose that for many databases, this won't be an issue.

   It might be useful to provide a way to specify a conversion function,
   although...

   - I think this is going to be prohibitively slow for large databases.
   The required downtime will be too long.

The analysis tool should be good for helping people find issues at the
application level.

For automated fixups, I think it would be better to work at the pickle
level.  I'm thinking of a tool that converts individual pickles. (This
might be useful for analysis as well.)

This would have a lot of advantages:

   - Potentially much faster because it doesn't require instantiating
   objects or running application-level code.
   - Doesn't require application classes to be around.
   - Can be integrated with existing workflows:
      - export/import
      - zodbcopy
      - ZRS replication.

I'll make a basic version of this that others can build on.

Jim

On Mon, May 23, 2016 at 9:47 AM, Michael Howitz <[email protected]> wrote:

> Hi,
>
> at "Berlin Strategic sprint 2016“ [1] Florian Pilz and me developed a tool
> to analyze a ZODB Filestorage for Python 2 string objects. When they are in
> an encoding besides ASCII this would prevent using this Filestorage with
> Python 3 because of decoding errors arising on loading the pickles.
> The tool is even able to convert those strings either to unicode by
> decoding them using a configurable encoding or convert them to
> `zodbpickle.binary` so Python 3 will read them as `bytes`.
> See [3] for the documentation of the tool and [2] for the repository on
> GitHub.
>
> * Is there already another tool for this analysis/migration?
> * Is there already any practical knowledge migrating Filestorage contents
> to Python 3?
> * Do you think such a tool is the right approach to achieve such a
> migration?
> * Is there anyone who wants to try out the tool on a Filestorage of a
> personal project and share the experiences? (I analyzed two projects where
> I have access to a Filestorage but I am sure this does not catch all the
> edge cases.)
>
>
>
> [1] http://gil.badall.net/2016/05/20/berlin-2016-sprint-update/
> [2] https://github.com/gocept/zodb.py3migrate
> [3] https://pythonhosted.org/zodb.py3migrate/
>
> --
> Michael Howitz · [email protected] · software developer
> gocept gmbh & co. kg · Forsterstraße 29 · 06112 Halle (Saale) · Germany
> http://gocept.com · Tel +49 345 1229889-8
> Python, Pyramid, Plone, Zope · consulting, development, hosting, operations
>
> --
> You received this message because you are subscribed to the Google Groups
> "zodb" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.