Re: packing directory storage

Toby Dickenson wrote:
> On Tuesday 22 June 2004 22:13, Richard Waid wrote:
> 
>>This reminded me, is it possible to tweak the logging
>>code slightly to 'checkpoint' every so often ... either every few
>>minutes or every 10,000 objects packed, or something. Basically, when
>>you're packing 5,000,000+ objects (we are), it takes a very long time
>>between packing stages, which, other than the fact that CPU time is
>>being consumed, makes it very difficult to judge if things are
>>progressing when dealing with it remotely.
> 
> I understand 'checkpoint' to mean that the packing process could be 
> interrupted and resumed at a later time. That is certainly possible, but its 
> not trivial. I cant see it happening without a volunteer or sponsor.

That would be cool, but isn't actually what I meant :)

> Or do you just want a log of progress? That is very much easier. But what is 
> the advantage over checking IO or cpu load?

That's what I meant. Not necessarily a log of actual progress, just an 
"I'm still alive" message. Actual progress would be great though.

The advantage over checking IO or CPU load is that a pack of our system 
takes around 3-5 hours, in a quiet time. The paranoid (like me) might 
wonder if the pack is actually proceeding, rather than just spinning. 
Just sugar really.

It might also help us to make more intelligent decisions about whether 
to let the pack proceed (if it's not likely to finish in time for the 
next period of load).


--Richard


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com