Re: Packing, again

On Mon, 2004-09-06 at 08:10 +0100, Toby Dickenson wrote:
> On Sunday 05 Sep 2004 22:29, Richard Waid wrote:
> > Hmm. As previously discussed (and to be remedied in all future packs
> > here), I didn't actually keep history, so how would I remedy using this?
> 
> the history I refer to above is the same sort of history that FileStorage 
> packing keeps, that you set using Zope's gui "keep X days of history". 
> history that can be seen using undo.
> 
> _not_ delay_delete history.

That was what I understood you to mean. The website itself is very much
in-production, history isn't usually important, and I've been packing
with 0 days of history up till now. Now that I understand how it can
help with my packing (as in, allowing the replicas to continue working),
I'll be packing with at least a few days of history from now on.

> > That was my understanding. Perhaps the behaviour of delay_delete should
> > be changed (for robustness) to the same as the normal case?
> 
> Suppose that was moved to pass 5 - what robustness would you gain? 
> Interrupting pass 4 would still cause the problem that has affected you,  but 
> you would have the option of renaming the -deleted files. In any case that 
> problem could be also fixed by restarting the pack and allowing it to finish, 
> as I describe above. The cost is in time - pass 5 would take as long as pass 
> 4 does today.

The option of renaming the -deleted files seems (to me at least) to be
an easier (if not better) option than rerunning a pack on a very large
storage. Question: could checkds have looked for a -deleted file to
'fix' the problem I hit automatically?

> Maybe a better option would be for pass 4 to determine which files need 
> deleting, but to not actually delete them. That deletion list could get 
> replayed before a subsequent pack. That should eliminate this problem, but 
> does suffer from scalability problems for packs that remove lots of files.

I see a possible pass 5 as a 'helper pass', as in, all it does is the
equivalent of "find -iname '*-deleted' | xargs -n1 rm -f", which you
could do yourself anyway. Pass 5 would only happen in the case where
delay_delete=0.

Oddly enough, this all gives me even _more_ faith in the work/code
you've put into directory storage.


--Richard




-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click