Re: Packing, again

On Sun, 2004-09-05 at 10:50 +0100, Toby Dickenson wrote:
> On Saturday 04 Sep 2004 23:18, Richard Waid wrote:
> > I can't find any 'deleted' transactions, which I'm assuming (and the
> > code backs this up) means that if you have 'delay_delete: 0' the
> > transactions are deleted in a first sweep. I should have checked the
> > code first ... I guess I assumed that if you had delay_delete:0 it would
> > rename then perform another sweep to remove.
> 
> Pass 4 simultaneously:
> 1. renames files to -deleted if the mark/sweep logic has found them to be 
> unnecessary, and 
> 2. unlinks -deleted files that are older than the delete_delay.
> 
> or, if delete_delay is zero, it unlinks them direct.

That was my understanding. Perhaps the behaviour of delay_delete should
be changed (for robustness) to the same as the normal case?

> > So... what does this mean? I'm assuming the missing files were removed
> > in the aborted pack.
> 
> If I understand you right, the interrupted pack was using permissions not 
> minimal?

Correct. I'm fairly confident that Minimal works pretty nicely, since
I've packed 1 smallish and 1 largish ZODB with it, with spectacular
results -- if you ignore the unlink time afterwards, it isn't that much
slower than a FileStorage pack.

> If this current problem was caused by the interruption then that interrupted 
> pack must have got as far as pass 4. That is right at the end of the packing 
> process. Please can you confirm this from the event log? 
> 
> If it didnt get that far then I am confident that the interruption wouldnt 
> have caused a problem

2004-08-29T01:37:12 INFO(0) DirectoryStorage Entered snapshot mode
'packing'
2004-08-29T01:37:12 INFO(0) DirectoryStorage Starting to pack
2004-08-29T01:37:12 INFO(0) DirectoryStorage Packing pass 1 of 4
2004-08-29T11:05:38 INFO(0) DirectoryStorage Packing pass 2a of 4
2004-08-29T12:29:53 INFO(0) DirectoryStorage Packing pass 2b of 4
2004-08-29T12:29:53 INFO(0) DirectoryStorage Packing pass 3 of 4
2004-08-29T12:35:40 INFO(0) DirectoryStorage Packing pass 4 of 4
2004-08-29T17:20:43 INFO(0) Z2 Caught signal SIGTERM

This also nicely illustrates the reason I was switching to a different
pack mechanism :) Note the 'consistency' pass, Pass 1, is instantaneous
(as expected) with Minimal, which saves about 10 1/2 hours :)

> > I'm also assuming that if they were deleted they 
> > probably weren't needed anyway (which explains while our application
> > hasn't turned into a steaming mess yet :)).
> 
> Thats an interesting observation. I'm very confident that an interrupted pack 
> wont corrupt the current version of the data, and pretty confident that it 
> wont cause problems when looking at historical revisions of objects, or doing 
> an undo etc. But there could still be a corner case bug where an interrupted 
> pack can cause trouble for a subsequent pack. You might not be any data 
> corruption at all.... 

How many files get unlinked for each transaction in a pack? Is it
possible that only one part of the 'transaction' got deleted, confusing
the pack? If so, moving the files to -deleted before deletion might have
got me out of trouble.

> I will think about this. And I could really use a traceback.

I'll send you this off-list.

> > I'm guessing I could fix 
> > this by copying the transactions from the replica, though that could be
> > pretty tedious.
> 
> You can get a full directory listing of master and replica with
> find /path/to/my/storage/A -type f -print 
> A diff between the two will show which files exist on the replica but not the 
> master.

The problem isn't one of figuring out the differing files, the problem
is that doing a 'find' anything on several million files takes a long
while, especially with a loaded system :)


--Richard



-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click