Re: Help with ZODB error in Groupserver installation please.

On Tue, Nov 1, 2016 at 12:12 AM, Peter Cheney <[email protected]> wrote:

> Hi Everyone,
>
> Have just joined your group as I really need a hand to fix an error that I
> am getting in a GroupServer installation. It happens after an upgrade from
> GroupServer 15.11 to 16.04. I've already been in contact with GroupServer
> list and have been referred to the ZODB folk to see if anyone here knows
> what might be the go. Admins, please redirect me if if posted incorrectly.
>

> From what I've seen there is some post upgrade linkage issue between the
> GroupServer application data and the ZODB database (exists within
> PostgreSQL) in that messages with attachments posted to the group are not
> able to be displayed in the browser. Other mail groups are unaffected as
> are new ones created post upgrade.
>
> The message I get in the browser is:
>
> Technical Description: 500 Internal Server Error
>
> URL: http://lists.itp.qld.gov.au/groups/nrm-crew/messages/
> topic/10SxfH0ivMzrRK2SJuiMij
>
> Issue: KeyError: '_all_columns'
> Attached is a complete log for when the error happens but here is an
> excerpt for starters.
>
> ------
> 2016-11-01T08:50:54 ERROR ZODB.Connection Couldn't load state for 0x07cd
> Traceback (most recent call last):
>   File "/app/groupserver-14.11/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/Connection.py",
> line 860, in setstate
>     self._setstate(obj)
>   File "/app/groupserver-14.11/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/Connection.py",
> line 914, in _setstate
>     self._reader.setGhostState(obj, p)
>   File "/app/groupserver-14.11/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/serialize.py",
> line 612, in setGhostState
>     state = self.getState(pickle)
>   File "/app/groupserver-14.11/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/serialize.py",
> line 605, in getState
>     return unpickler.load()
>   File "/app/groupserver-14.11/eggs/SQLAlchemy-0.9.10-py2.7-linux-
> x86_64.egg/sqlalchemy/sql/base.py", line 584, in __setstate__
>     self.__dict__['_all_columns'] = state['_all_columns']
> KeyError: '_all_columns'
>


To state the obvious, this version expects state to have data that's not
there any more.   This appears to be a clear-cut change in object
serialization as part of the upgrade.  This looks like an SQLAlchemy
change, which is perhaps why the GroupServer folks turned you away.



> ------
> (NB the groupserver-14.11 is a hangover from the incorrect naming of the
> original installation directory of GroupServer)
>
> Not sure where the '_all_columns' key is supposed to exist and I cannot
> see when searching in the groupserverzodb database. My knowledge of Python
> is zilch and am really struggling to find the answers.
>

Ouch, then you're going to need to find someone to help you dig into this.



> Here is the schema:
> groupserverzodb=# \dt
>                 List of relations
>  Schema |       Name        | Type  |    Owner
> --------+-------------------+-------+-------------
>  public | blob_chunk        | table | gszodbadmin            0 rows
> zoid, tid, chunk_num, chunk
>  public | commit_lock       | table | gszodbadmin            0 rows    (no
> columns defined)
>  public | current_object    | table | gszodbadmin        12747 rows
> zoid, tid
>  public | object_ref        | table | gszodbadmin            0 rows
> zoid, tid, to_zoid
>  public | object_refs_added | table | gszodbadmin            0 rows    tid
>  public | object_state      | table | gszodbadmin        33879 rows
> zoid, tid, prev_tid, md5, state_size, state
>  public | pack_object       | table | gszodbadmin            0 rows
> zoid, keep, keep_tid, visited
>  public | pack_state        | table | gszodbadmin            0 rows
> tid, zoid
>  public | pack_state_tid    | table | gszodbadmin            0 rows    tid
>  public | transaction       | table | gszodbadmin         1782 rows
> tid, packed, empty, username, description, extension
> (10 rows)
>
> groupserverzodb=#
>
> When I create a new mail group and submit messages with attachments, all
> works as expected. It's just this one group that is broken and (of course)
> it's the one that is most used.
>
> There is obviously something wrong with the GroupServer-16.04 buildout
> because when I roll back the installation to the 15.11 version the issue
> disappears. I've tried all the GroupServer "buildout" permutations I can
> think of and nothing seems to rectify the problem.
>
> What I do not know is where to look and what to fix and am hoping someone
> here can point me in the right direction please.
>
> I've also seen and tried a utility called fsrecover.py but it fails with
> "ImportError: No module named ZODB" (this is with
> PYTHONPATH=/app/groupserver-14.11/eggs/Zope2-2.13.24-py2.
> 7.egg/Zope2/utilities/skel/lib/python which is the only place I can find
> a python lib dir.). I then tried a "pip install ZODB" which fixed the
> import ZODB issue but broke the groupserver install...
>


This isn't a file-storage issue anyway. fsrecover is for situations where a
file-storage database file is corrupted, typically due to bad hardware. (I
haven't seen this in a long time.)

This is a software issue, most likely with GroupServer or sqlalchemy.


> So I'm stuck for ideas now. Many thanks in advance for considering my
> plight.
>

I'm not familiar with GroupServer. Do they offer paid support? This is
probably going to take some digging.

You might ask on the sqlalchemy list if they have seen this error before.

My guess:

- Something changed in sqlalchemy, although this seems to be a very old
version.

- Some object from sqlalchemy got stored in the ZODB database that
shouldn't have been.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.