Re: packing directory storage

On Wednesday 23 June 2004 17:21, Stefano Maffulli wrote:
> On Mon, 2004-06-21 at 19:12, Toby Dickenson wrote:
> > On Monday 21 June 2004 16:45, Stefano Maffulli wrote:
> > > one little issue I still don't have clear regarding ds is: how to pack
> > > it?  the system used for data.fs does not work
> >
> > You dont say what the system is, but most likely it should do.
>
> I rephrase my question, since I think I have now isolated the problem:
> is there a way to pack the ds without having to use the web interface?
> Having run checkds.py without it reporting any errors on the ds, having
> successfully started the installation using a small data.fs, I came to
> the conclusion that a 25/30gb ds is just too much to handle and the
> server hangs for that very simple reason.

How badly is the web interface wedged? you might be able to trigger the pack 
method over http (there are various scripts on www.zope.org) even if it cant 
display the form in a browser....

To pack without Zope in the loop:

1. Shut down the zope or zeo process.
2. Run this script (untested; debugging might be necessary)

import time
from DirectoryStorage.Full import Full as S
from DirectoryStorage.Filesystem import Filesystem
from ZODB.referencesf import referencesf as ZODB_referencesf
fs = Filesystem('/var/lib/zope/var/dirstorage')
storage = S(fs)
# pack keeping 1 week of history
storage.pack(time.time()-60*60*24*7,ZODB_referencesf) 

I would be interested to hear any statistics on a storage of that size.... how 
long does it take to pack? how long does it take to run checkds? the output 
from checkds with the -v switch would be interesting too.


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com