Re: Packing, again

On Thu, 2004-09-02 at 05:55 +0100, Toby Dickenson wrote:
> On Thursday 02 Sep 2004 04:57, Richard Waid wrote:
> > Just a thought -- couldn't the mark/sweep be incremental anyway -- just
> > run the pack so it deletes, say, 3 days of transactions, then run it
> > again so it deletes 3 days of transactions etc. If used in combination
> > with a memory packing strategy, where you don't need to mark the file
> > permission bits for each file it should be quite quick (naive?)
> 
> To completely remove any object, you need to scan the *whole* storage to prove 
> that no other object has a reference to it. That's the only way to reclaim 
> space from 'deleted' objects.

I've just tried a pack of our largest dirstorage, running at somewhere
between 8-10 gig. It aborted with a 'FileDoesNotExist' error, which
worries me quite a bit.

"""
FileDoesNotExist: DirectoryStorage file 'A/o000/000/000/012/A5D6.c' does
not exist
"""

I can send the full traceback, but I'd need to trim the paths out first
(since it gives away too much about our local layout).

What could cause this? The missing file _does_ exist in our replica, and
interestingly enough the date seems to correspond exactly to where
someone accidently copied about 1.5GB of data to another part of the
ZODB, which we discovered and subsequently deleted a couple of days ago.
The database hasn't been packed since that event (neither the copying or
the deletion).

Perhaps more importantly, what can be done to fix it (without data
loss)? I'm currently exporting the entire database to a file (via the
export mechanism in ZODB) to mitigate against any possible dataloss.


--Richard



-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click