Re: File transfer speed slows 10x and more after reaching 1TB in size

On 02/11/2016 03:31 AM, alanb wrote:
> Update:
> 
> I reformatted one of the 4TB drives to XFS, and did another transfer of 
> the original 2TB image file from JFS to XFS.
> 
> What is interesting, is when I copy the 2TB image file from the JFS 
> drive to the XFS drive, I still encountered the same slow down. It was 
> doing OK at 1TB, but when I next checked on it, it was thrashing and 
> slow at the 1.3TB mark and it persisted until completion, so the problem 
> point starts somewhere between 1TB and 1.3TB. I spot checked the CPU 
> usage and it was OK the whole time.
> 
> My next test was extracting the file system from the image file on the 
> XFS drive to the free space on the same XFS drive (same partition), so 
> from XFS to XFS. When I last did the file system extraction, it was from 
> JFS to JFS, and it would slow down at around the 1TB mark (probably 
> between 1TB and 1.3TB), but this time it ran smoothly until completion. 
> I think this indicates that it is the reads from the large file are the 
> source of the problem.
> 
> I have not yet tested transferring from XFS to JFS, but hope to give it 
> a try once I find the time.
> 
> I think it is now very clear that JFS is the source of the problem, and 
> it's the reads that are not working properly starting somewhere between 
> 1-1.3TB.

I'm guessing that maybe at this point the block allocation might be
breaking down and the later part of the file may be badly fragmented. I
don't have time right now investigate. Just throwing out a theory.

> 
>  From now on, I'll be using XFS when dealing with very large files.

Might be a good idea. JFS hasn't gotten much attention in the last
several years. :-(

Thanks,
Dave

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140