session/otks gdbm db's not compatible - python3 conversion issue?

Hi folks:

I am looking for some help with anydbm/dbm files under python2/3.
These files are used for one time tickets (otks) and session databases
in roundup.

The default dbm format created by the anydbm module in python2 is the
dbhash format. That format is not supported in python 3 by the dbm
(the replacement for anydbm) module. Both anydbm and dbm support gdbm
and ndbm format files.

I thought I could use the python3 dbm module and create an ndbm file
or gdbm file. Then this could also be used by python 2.

However when I try to use the db's with roundup running under python2
I get a marshaling error. AFAICT python2 is able to manipulate the db
files. I can use:

> python2
Python 2.7.6 (default, Nov 13 2018, 12:45:42)
>>> import anydbm
>>> a=anydbm.open('demo/db/otks')
>>> type(a)
<type 'gdbm.gdbm'>
>>> a.keys()

I get the list of keys. So the db is properly opened and accessed.  My
guess is there is an issue with the data. Maybe byte vs. string in the
data payload?

Dumping one of the keys I get:

>>> a['QGFLUSQYWJHH5JSHFROOOPVABRRSZJ2QC4FDV6IDOEGSOTF6C5Q3MZBP3OLRXBM2']
'\xfb\xda\x03uid\xda\x011\xda\x0b__timestamp\xe7w\xcaq\xfc\xe9%\xd7A\xda\x03sid\xfa+EAULfpfOAkobeda8aQSz1ogLV4rlwkCoplUZIkgkJ1Y0

So I can access the data at least.

The rdbms backends seem to be usable with both python versions. I
wonder if this will also be an issue for using the anydbm backend
rather than sqlite/mysql etc.

While this isn't a disaster, upgrading would be disruptive requiring
people to log in again, or perform incomplete password resets/signups
over again.

The marshaling traceback I get is:
 
======
ValueError: bad marshal data (unknown type code) Python 2.7.6 /usr/bin/python

A problem occurred while running a Python script. Here is the sequence
of function calls leading up to the error, with the most recent
(innermost) call first.

The exception attributes are:
      __class__ = <type 'exceptions.ValueError'> 
      __delattr__ = <method-wrapper '__delattr__' of exceptions.ValueError object> 
      __dict__ = {} 
      __doc__ = 'Inappropriate argument value (of correct type).' 
      __format__ = <built-in method __format__ of exceptions.ValueError object> 
      __getattribute__ = <method-wrapper '__getattribute__' of exceptions.ValueError object> 
      __getitem__ = <method-wrapper '__getitem__' of exceptions.ValueError object> 
      __getslice__ = <method-wrapper '__getslice__' of exceptions.ValueError object> 
      __hash__ = <method-wrapper '__hash__' of exceptions.ValueError object> 
      __init__ = <method-wrapper '__init__' of exceptions.ValueError object> 
      __new__ = <built-in method __new__ of type object> 
      __reduce__ = <built-in method __reduce__ of exceptions.ValueError object> 
      __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.ValueError object> 
      __repr__ = <method-wrapper '__repr__' of exceptions.ValueError object> 
      __setattr__ = <method-wrapper '__setattr__' of exceptions.ValueError object> 
      __setstate__ = <built-in method __setstate__ of exceptions.ValueError object> 
      __sizeof__ = <built-in method __sizeof__ of exceptions.ValueError object> 
      __str__ = <method-wrapper '__str__' of exceptions.ValueError object> 
      __subclasshook__ = <built-in method __subclasshook__ of type object> 
      __unicode__ = <built-in method __unicode__ of exceptions.ValueError object> 
      args = ('bad marshal data (unknown type code)',) 
      message = 'bad marshal data (unknown type code)'

.../roundup/backends/sessions_dbm.py in getall(self=<roundup.backends.sessions_dbm.Sessions instance>, infoid='EAULfpfOAkobeda8aQSz1ogLV4rlwkCoplUZIkgkJ1Y')
   78         try:
   79             try:
   80                 d = marshal.loads(db[infoid])
      d = undefined, global marshal = <module 'marshal' (built-in)>, global loads = undefined, db = <gdbm.gdbm object>, infoid = 'EAULfpfOAkobeda8aQSz1ogLV4rlwkCoplUZIkgkJ1Y'
   81                 del d['__timestamp']
   82                 return d

.../roundup/cgi/client.py in __init__(self=<roundup.cgi.client.Session instance>, client=<roundup.cgi.client.Client instance>)
  171             else:
  172                 self._sid = cookies[self.cookie_name].value
  173                 self._data = self.session_db.getall(self._sid)
      self = <roundup.cgi.client.Session instance>, global _data = undefined, global session_db = undefined, global getall = undefined, global _sid = undefined
  174 
  175     def _gen_sid(self):
...
=====

when trying to initialize a Session object for a logged in user.

Any quips, comments, evasions, questions or answers on this?

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.