Re: Re: Python-3-Migration of existing Data.fs

There it is : https://github.com/vert-rouge/sample-zodb-migration-2to3
comments are in French, sorry about that.

I would like to add that if you make a subtle change in ZODB._compat.py, my
problem is solved.

around line 50
    class Unpickler(zodbpickle.pickle.Unpickler):
        def __init__(self, f):
-            super(Unpickler, self).__init__(f)
+            super(Unpickler, self).__init__(f, errors="bytes")


It’s done for loads around line 70
    def loads(s):
        return zodbpickle.pickle.loads(s, encoding='ASCII', errors='bytes')


Regards,

Arthur


Arthur

2018-03-20 14:06 GMT+00:00 Jim Fulton <[email protected]>:

>
>
> On Tue, Mar 20, 2018 at 8:02 AM, Arthur LEDARD <[email protected]>
> wrote:
>
>> I have a sample code to reproduce the problem.
>>
>
> Excellent!
>
>
>> I can share maybe on github?
>>
>
> Sure, or in a gist, or whatever. :)
>
>
>>
>> I’m afraid I’m not explicitely using oid, just an old good OOBTree,
>> containing OOBTree.
>>
>
> Hm.  I look forward to playing with your sample code this weekend!!!
>
> Jim
>
>
>>
>>
>> Arthur
>>
>> 2018-03-20 13:51 GMT+00:00 Jim Fulton <[email protected]>:
>>
>>> On Tue, Mar 20, 2018 at 6:22 AM, Arthur LEDARD <[email protected]>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I’m trying to migrate my Data.fs.
>>>> I use your code, but I’m stuck with a non expected problem.
>>>>
>>>> Using python2, my oid are pickled in a format
>>>> "U\x08\x00\x00\x00\x00\x00\x00\x03\xeaq\x03"
>>>> The format seems correct, but is not understood with python3 :
>>>> UnicodeDecodeError.
>>>>
>>>
>>> Ouch.
>>>
>>>
>>>>
>>>> I was expecting to have problems with data, not oid.
>>>>
>>>
>>> I assume you're storing oids in your data somewhere.  Can you confirm?
>>>
>>> I happen to like this pattern (as opposed to generating ids at the
>>> application level), but I can see how this might be a headache in the
>>> transition to Python 3.
>>>
>>> Did you face this problem?
>>>> Did you succeed in your migrations?
>>>>
>>>
>>> I suspect they were focussed on providing a tool for future migrations,
>>> but am curious about any experience they might have.
>>>
>>> Jim
>>>
>>>
>>>>
>>>> Regards,
>>>>
>>>> Arthur
>>>> Le lundi 23 mai 2016 16:48:18 UTC+1, mh a écrit :
>>>>>
>>>>> Hi,
>>>>>
>>>>> at "Berlin Strategic sprint 2016“ [1] Florian Pilz and me developed a
>>>>> tool to analyze a ZODB Filestorage for Python 2 string objects. When they
>>>>> are in an encoding besides ASCII this would prevent using this Filestorage
>>>>> with Python 3 because of decoding errors arising on loading the pickles.
>>>>> The tool is even able to convert those strings either to unicode by
>>>>> decoding them using a configurable encoding or convert them to
>>>>> `zodbpickle.binary` so Python 3 will read them as `bytes`.
>>>>> See [3] for the documentation of the tool and [2] for the repository
>>>>> on GitHub.
>>>>>
>>>>> * Is there already another tool for this analysis/migration?
>>>>> * Is there already any practical knowledge migrating Filestorage
>>>>> contents to Python 3?
>>>>> * Do you think such a tool is the right approach to achieve such a
>>>>> migration?
>>>>> * Is there anyone who wants to try out the tool on a Filestorage of a
>>>>> personal project and share the experiences? (I analyzed two projects where
>>>>> I have access to a Filestorage but I am sure this does not catch all the
>>>>> edge cases.)
>>>>>
>>>>>
>>>>>
>>>>> [1] http://gil.badall.net/2016/05/20/berlin-2016-sprint-update/
>>>>> [2] https://github.com/gocept/zodb.py3migrate
>>>>> [3] https://pythonhosted.org/zodb.py3migrate/
>>>>>
>>>>> --
>>>>> Michael Howitz · [email protected] · software developer
>>>>> gocept gmbh & co. kg · Forsterstraße 29 · 06112 Halle (Saale) ·
>>>>> Germany
>>>>> <https://maps.google.com/?q=Forsterstra%C3%9Fe+29+%C2%B7+06112+Halle+(Saale)+%C2%B7+Germany&entry=gmail&source=g>
>>>>> http://gocept.com · Tel +49 345 1229889-8 <+49%20345%2012298898>
>>>>> Python, Pyramid, Plone, Zope · consulting, development, hosting,
>>>>> operations
>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "zodb" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> --
>>> Jim Fulton
>>> http://jimfulton.info
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "zodb" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Jim Fulton
> http://jimfulton.info
>

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.