Re: db size problem (4GB monster)

Hello Jörn,
it seems that the main-problem ist the file-system which is not able to
handle file-sizes over 4GB.

There is only two ways you can do:

1. find a other filesystem that can handle such big files and there you can
reduce it for example by packing the data.fs (i do not know such a
file-system)

2. there is a tool (i do not know at the moment the name) that is able to
open that big file and then you delete a big block of data/content out the
the viewable informations you see, YES it sounds bad but the problem is also
bad. It is like open the file in a text-editor and deleting text!

I had the same problem and this was the only way to bring the zope-server
back to run. After you delete some data out of the open file you start the
zope-server and inside there is a script that recognize automatically that
the file is inconsistent and repaired it.

That was the way i fix the problem.

Hope you will find a solutionfor this bad problem.

Regards

Richard



2008/10/1 Jörn Paessler <[email protected]>

>   Hi Tobias,
>
> thank you very much for this hint.
>
> Unfortunately this didn't really help since the python process quits
> with "Segmentation fault" after 10 minutes of processing.
>
> Any other ideas?
>
> Why can't I re-import an XML file which was exported by the same
> instance?
>
> Best
> Jörn
>
>
> ----------------------------------------------------------
> beyond content GmbH
> Dipl.-Ing. Jörn Paessler
> Geschäftsführer
> Burgschmietstr. 10
> 90419 Nürnberg, Germany
> E-Mail: [email protected]<joern.paessler%40beyond-content.de>
> Web: www.beyond-content.de
> Fon: +49 (0)911 977 98162
> Fax: +49-(0)911 787 2525
> Geschäftsführer: Dipl.-Ing. Jörn Paessler
> Sitz der Gesellschaft: Nürnberg
> Handelsregister: Amtsgericht Nürnberg HRB 23740
> USt-IdNr.: DE247571538
>
> Am 30.09.2008 um 18:03 schrieb Tobias Greitzke:
>
>
> > Hi Jörn,
> >
> > we had the same problem last year. The instance doubles it's size with
> > every move you make like copying objects. We couldn't determine where
> > this was coming from, but the refactoring script written by Mr
> > Hoffmann
> > helped us getting it fixed.
> >
> >
> http://www.zms-publishing.com/support/content/e724/e727/e1456/index_ger.html
> >
> > Good Luck,
> > Tobias
> >
> > Jörn Paessler schrieb:
> > >
> > >
> > > Hi ZMS devs,
> > >
> > > we have a big problem with an ZMS site and DB size.
> > >
> > > Here are the facts:
> > > - XML-export including binaries has a size of 100 MB
> > > - the ZODB used to have a size of 400-500 MB
> > > - suddenly, a few weeks ago, the size of the ZODB grew to twice the
> > > size (1 GB)
> > > - that happened another two times, today the DB has a size of 4 GB
> > > - packing ZODB has no effect
> > > - trash is empty
> > > - catalog is not active
> > > - cache is not active
> > > - the site does not contain large binary files or downloads, just
> > > several files < 1MB
> > > - next strange behavior: exporting different nodes (level 1)
> > produces
> > > 4 GB .zexp files for all single nodes
> > > - exporting as XML and importing it again into a naked site
> > results in
> > > the error message "The currently logged-in user does not have the
> > Copy
> > > or Move permission respective to the object zms."
> > > - the only way to reduce the size of the DB is to delete one node by
> > > the other, pack and if lucky the "monster" gets found: a very time
> > > consuming procedure.
> > >
> > > Did anybody make the same experience?
> > > Does anybody have an idea what might gone wrong?
> > >
> > > Thanks a lot for your help.
> > >
> > > Best
> > > Joern Paessler
> > >
> > > Our setup:
> > > ZMS: ZMS 2.10.5-06 (Build #129f)
> > > ZOPE: (Zope 2.10.5-final, python 2.4.3, darwin)
> > > PYTHON: 2.4.3 (#1, Oct 8 2006, 23:00:57) [GCC 4.0.1 (Apple Computer,
> > > Inc. build 5341)]
> > > System Platform: darwin
> > > SOFTWARE_HOME: /Applications/Zope/Zope-2.10.5/lib/python
> > > ZOPE_HOME: /Applications/Zope/Zope-2.10.5
> > > INSTANCE_HOME: /Users/jp/data/zope/instance-8004
> > > CLIENT_HOME: /Users/jp/data/zope/instance-8004/var
> > > System Time: 2008/09/30 17:00:37.711 GMT+2
> > > Running For: 6 hours 26 sec
> > > ZODB Location: /Users/jp/data/zope/instance-8004/var/Data.fs
> > > Database Size: 4187.9M
> > >
> > > ----------------------------------------------------------
> > > beyond content GmbH
> > > Dipl.-Ing. Jörn Paessler
> > > Geschäftsführer
> > > Burgschmietstr. 10
> > > 90419 Nürnberg, Germany
> > > E-Mail: [email protected]<joern.paessler%40beyond-content.de>
> > > <mailto:joern.paessler%40beyond-content.de<joern.paessler%2540beyond-content.de>
> >
> > > Web: www.beyond-content.de
> > > Fon: +49 (0)911 977 98162
> > > Fax: +49-(0)911 787 2525
> > > Geschäftsführer: Dipl.-Ing. Jörn Paessler
> > > Sitz der Gesellschaft: Nürnberg
> > > Handelsregister: Amtsgericht Nürnberg HRB 23740
> > > USt-IdNr.: DE247571538
> > >
> > >
> >
> >
>
>  
>