Packing, again

I'm currently running several dirstorage instances, pretty much
flawlessly I might add. The only problem is packing -- one of our
instances is currently using 8-10 gig of storage (it's actually hard to
measure exactly, because doing a du takes a very, very long time on
reiser3 for some reason :)). It takes well over 12 hours to do _stage 1_
of a pack cycle, which is very bad when you need to pack, but have a
busy site. I've had to abort the last couple of attempts because it was
having a significant impact on site performance, even though the pack
_started_ well outside of peak load hours.

I'm looking for an alternative here. The MinimalStorageMarker method
claims to be between 20% and 80% faster -- is it just as stable? I note
that I could just create a 'misc/packing' and link it to a different
drive.

It would also really, really help to have some kind of 'checkpoint'
messages in the packing process. At some stages in the packing it's
really difficult to tell if it's actually doing something, if it has
locked up, or if it is still going. Maybe every 10,000 transactions it
could just write something like 'marked x0,000 transactions, last was:
xxxxxx'. If there is some way to figure out _actual_ progress that would
be even better.

Another issue is replicas. Once the storage has finished packing, I need
to re-tar it, rsync it, then resume replication, which means there is a
significant period of time when the replica is out of sync. Is there a
better way to do this?

Any help or suggestions would be greatly appreciated.

--
Richard Waid
Network/Software Engineer
http://iopen.net



-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click