Re: Slow Folder Traversal on One system, Fast on Another

On Mon, 18 Jan 2010 19:13:54 -0800
Tanthrix <[email protected]> wrote:

> Jeff Layton wrote:
> > Ok, good to check. Other than the change to server inode numbers, I'm
> > not sure of any huge changes to the readdir code recently. You could
> > try mounting with '-o nounix' to disable unix extensions. I'm not sure
> > what the state of those was around 2.6.21.
> >
> > The more you can do to narrow it down, the better. If you can rule out
> > hw changes, then you might get some wire captures while doing this on
> > the old and new kernels and look for the differences there.
> >
> >   
> So here's what I did:
> 
> I created a new share called "Test" on my Windows7 machine. This 
> directory contains 10 subdirectories, each with some 800+ blank text 
> files in it. The total number of folders is 10, and the total number of 
> files is 8,270. I figured this would help exaggerate the differences in 
> speed, making it a bit easier to see what was going on.
> 
> I then mounted this share on my Ubuntu 2.6.31-17 machine and my older 
> Arch Linux 2.6.21 machine using the exact command "mount -t cifs -o 
> username=q,password=q //192.168.0.2/test /Test". I then performed an "ls 
> -R" on the /Test directory on both machines while running a wire capture.
> 
> Here are the results with links to SMB-filtered tcpdump summaries as 
> well as the captures with full packets:
> 
> Ubuntu 2.6.31-17
> ------------------
> Time to complete: ~18 seconds
> Capture Summary - 4.95mb: http://novae-res.org/sambatest/Ubuntu.txt
> Full dumpfile  - 9.15mb: http://novae-res.org/sambatest/UbuntuTcpdump
> 
> Arch Linux 2.6.21
> ------------------
> Time to complete: ~2 seconds
> Capture Summary - 416kb: http://novae-res.org/sambatest/Arch.txt
> Full dumpfile - 470kb: http://novae-res.org/sambatest/ArchTcpdump
> 
> Clearly, the Ubuntu machine is doing a whole heck of a lot more, but I 
> must admit that I'm in over my head here, so I can't draw any useful 
> conclusions or solutions from the above data.
> 
> Any thoughts you have will of course be appreciated, and again I thank 
> you kindly for taking the time to help me with this.

The two kernels are using different infolevels for the FIND_FIRST call
(not too surprising since this code has changed significantly in 10
releases).

The main difference though is that the later kernel is doing a
QPathInfo against all of the inodes, and the earlier one isn't. The
readdir code doesn't trigger this, so I suspect that the time
difference is due to differences in userspace. i.e., the later release
appears to be doing more than just readdir() type calls when you call
'ls -R'.

One question...when you do the "ls" on ubuntu, is the output colorized?
If so, can you run:

    $ unalias ls

...and then in the same shell, redo your 'ls -R' test on the ubuntu
box. Does it go faster?

-- 
Jeff Layton <[email protected]>