Re: File system defragmentation. Something that can resume after stopping.

On 5/13/26 5:47 AM, Michael wrote:
> On Tuesday, 12 May 2026 23:45:25 British Summer Time Dale wrote:
>> On 5/12/26 6:02 PM, Jack wrote:
>>> On 2026.05.12 18:45, Dale wrote:
>>>> Howdy,
>>>>
>>>> As most likely know, I have some fairly large storage systems here.
>>>> I
>>>> have one that while doing a fsck on, I noticed that it is really
>>>> fragmented.  Something around 40% or so.  I'm running e4defrag on
>>>> it
>>>> but I suspect something will cause me to have to stop it before it
>>>> finishes.  It has over 50TBs of data on it.  It's going to take a
>>>> while.  I've looked at the man page and it has no resume option or
>>>> many options at all really.  One to test, one to defrag is really
>>>> it.  Rest is visual options like verbose.
>>>>
>>>> I've googled around looking to see if someone came up with a
>>>> script/tool or something that can defrag a file system and can
>>>> resume
>>>> if needed.  I can't find one.  Given e4defrag doesn't have good way
>>>> to do it, I'm not surprised really.  The only thing I can come up
>>>> with is to create a list of files that needs to be defraged, and
>>>> then
>>>> have a script that does them one by one until it is done. May
>>>> require
>>>> running more than once to really get the job done.
>>>>
>>>> I have other file systems that are doing a lot better than this one
>>>> but I also understand why this one is being fragmented so much.
>>>> This
>>>> file system receives files that are not complete but is being
>>>> constantly written to until over time, it does complete.  This file
>>>> system is on the receiving end of a disaster for keeping files in
>>>> one
>>>> long write.  I'm sure ext4 is doing the best it can but it can only
>>>> do so much.  I doubt any file system can do any better.
>>>>
>>>> Does anyone know of a tool to use that has a resume option and
>>>> works
>>>> on ext4?  Obviously, I'd like it to be something that is well
>>>> tested
>>>> and known to be safe.  I'm just not having any luck finding one
>>>> myself.  Maybe one doesn't exist.
>>> Why not just start it again?  It isn't going to defrag any files it
>>> already defragged, is it?  As long as interrupting it is safe, and
>>> doesn't ever leave a damaged file or file system, I can't think of
>>> any problems.
>>>
>>> Jack
>> Actually, it starts over with the first file and starts working its
>> way through again.  At least that is what it did in the past.  It
>> puts this in front of each file.  [1/162725]  It starts at file 1 and
>> then works its way through until it has done each file.  As you can
>> see, I have over 162,000 files.  Right now, it is on file 49,566.
>> It's been running since Saturday I think.  Basically, it rewrites
>> each file from 1 to total number of files whether it is fragmented or
>> not.  That's what it looks like to me at least.
>>
>> I will say that, in the past, it seems it is faster on the second run
>> at first because it doesn't actually move data as much for file it
>> already finished but it still checks each file and it is going to
>> take a while still.  It doesn't save much time since it still checks
>> every file in sequence.  Once it gets to where it stopped on first
>> run, it slows down again as it moves data more.
>>
>> I'll be honest, whatever it is would either have to use something
>> other than e4defrag or create a list in a file somewhere then figure
>> out which file to do first and do them one at a time with e4defrag.
>> I'd suspect, a completely different tool would be the best way.  I'm
>> just not sure if anyone has even done this.  Most people most likely
>> don't even check to see if their files are fragmented or not.  Given
>> the cost of hard drives right now, I hope one of these drives doesn't
>> go bad.  I have one 20TB and one 12TB or 14TB spare but that is it.
>> I try to keep a spare around just in case.
>>
>> I'll be glad when the AI folks are done buying hard drives.  They have
>> driven the cost of hard drives and other computer components through
>> the roof.  Given the number of drives in my rig right now, I'm
>> sitting on a small fortune.  ;-)
>>
>> Dale
>>
>> :-)  :-)
> People have written scripts to fsck, optimize, defragment ext4 fs, but
> I'm thinking it could be ext4 is less suited to your particular use
> case.
>
> BTRFS has a defrag command which can defrag recursively individual
> directories, as well as start from a given offset.
>
> However, XFS is probably a more suitable fs with its defrag command.  It
> starts defragging the top 10% of files with the highest number of
> extents.  After two hours it saves its current status in /var/
> tmp/.fsrlast_xfs and will start from this point onward next time you re-
> run the command.  The default time is -t 7200 (2 hours in seconds), but
> you can set this to be as long as desired.


Well, as it is, I'm kinda stuck.  I'd have to buy at least 3 or 4 hard 
drives to put a different file system on and then transfer everything 
over.  Given the price of hard drives, I don't plan to buy any hard 
drives any time soon.  Plus, for most of my uses, ext4 does fine.  The 
biggest thing on this one file system, the files are always being added 
to and plus new stuff being added.  My other file systems, I copy files 
over but they are all done with one long write since they are complete 
which makes them more likely to not be fragmented.  Once they there, 
they don't change anymore.

I was just hoping someone with some serious scripting skills had 
invented this wheel already.  If they have, it sure is well hidden. Me 
and Duck Duck Go couldn't find it and it seems no one else is aware of 
it either.  LOL

I'll just have to leave it running and maybe skip updates this weekend.  
Maybe it will finish by next weekend.  I should have ran it within 
screen.  I just wonder how long it will stay with the lower amount of 
fragmentation.  It's not like it isn't going to continue to change a 
lot.  Heck, it's adding about 5 to 6MB/s as I type.  By the time it 
finishes running, it may need to be run again.  :/

Well, it was a thought.

Dale

:-)  :-)