Re: File system defragmentation. Something that can resume after stopping.

Michael <[email protected]> Wed, 13 May 2026 11:47:39 +0100
Newsgroups	gmane.linux.gentoo.user
Message-ID	<1881248.TLkxdtWsSY@rogueboard>
On Tuesday, 12 May 2026 23:45:25 British Summer Time Dale wrote:
> On 5/12/26 6:02 PM, Jack wrote:
> > On 2026.05.12 18:45, Dale wrote:
> >> Howdy,
> >> 
> >> As most likely know, I have some fairly large storage systems here.
> >> I
> >> have one that while doing a fsck on, I noticed that it is really
> >> fragmented.  Something around 40% or so.  I'm running e4defrag on
> >> it
> >> but I suspect something will cause me to have to stop it before it
> >> finishes.  It has over 50TBs of data on it.  It's going to take a
> >> while.  I've looked at the man page and it has no resume option or
> >> many options at all really.  One to test, one to defrag is really
> >> it.  Rest is visual options like verbose.
> >> 
> >> I've googled around looking to see if someone came up with a
> >> script/tool or something that can defrag a file system and can
> >> resume
> >> if needed.  I can't find one.  Given e4defrag doesn't have good way
> >> to do it, I'm not surprised really.  The only thing I can come up
> >> with is to create a list of files that needs to be defraged, and
> >> then
> >> have a script that does them one by one until it is done. May
> >> require
> >> running more than once to really get the job done.
> >> 
> >> I have other file systems that are doing a lot better than this one
> >> but I also understand why this one is being fragmented so much.
> >> This
> >> file system receives files that are not complete but is being
> >> constantly written to until over time, it does complete.  This file
> >> system is on the receiving end of a disaster for keeping files in
> >> one
> >> long write.  I'm sure ext4 is doing the best it can but it can only
> >> do so much.  I doubt any file system can do any better.
> >> 
> >> Does anyone know of a tool to use that has a resume option and
> >> works
> >> on ext4?  Obviously, I'd like it to be something that is well
> >> tested
> >> and known to be safe.  I'm just not having any luck finding one
> >> myself.  Maybe one doesn't exist.
> > 
> > Why not just start it again?  It isn't going to defrag any files it
> > already defragged, is it?  As long as interrupting it is safe, and
> > doesn't ever leave a damaged file or file system, I can't think of
> > any problems.
> > 
> > Jack
> 
> Actually, it starts over with the first file and starts working its
> way through again.  At least that is what it did in the past.  It
> puts this in front of each file.  [1/162725]  It starts at file 1 and
> then works its way through until it has done each file.  As you can
> see, I have over 162,000 files.  Right now, it is on file 49,566. 
> It's been running since Saturday I think.  Basically, it rewrites
> each file from 1 to total number of files whether it is fragmented or
> not.  That's what it looks like to me at least.
> 
> I will say that, in the past, it seems it is faster on the second run
> at first because it doesn't actually move data as much for file it
> already finished but it still checks each file and it is going to
> take a while still.  It doesn't save much time since it still checks
> every file in sequence.  Once it gets to where it stopped on first
> run, it slows down again as it moves data more.
> 
> I'll be honest, whatever it is would either have to use something
> other than e4defrag or create a list in a file somewhere then figure
> out which file to do first and do them one at a time with e4defrag.
> I'd suspect, a completely different tool would be the best way.  I'm
> just not sure if anyone has even done this.  Most people most likely
> don't even check to see if their files are fragmented or not.  Given
> the cost of hard drives right now, I hope one of these drives doesn't
> go bad.  I have one 20TB and one 12TB or 14TB spare but that is it. 
> I try to keep a spare around just in case.
> 
> I'll be glad when the AI folks are done buying hard drives.  They have
> driven the cost of hard drives and other computer components through
> the roof.  Given the number of drives in my rig right now, I'm
> sitting on a small fortune.  ;-)
> 
> Dale
> 
> :-)  :-)

People have written scripts to fsck, optimize, defragment ext4 fs, but 
I'm thinking it could be ext4 is less suited to your particular use 
case.

BTRFS has a defrag command which can defrag recursively individual 
directories, as well as start from a given offset.

However, XFS is probably a more suitable fs with its defrag command.  It 
starts defragging the top 10% of files with the highest number of 
extents.  After two hours it saves its current status in /var/
tmp/.fsrlast_xfs and will start from this point onward next time you re-
run the command.  The default time is -t 7200 (2 hours in seconds), but 
you can set this to be as long as desired.
signature.asc (application/pgp-signature, 870 B)