Re: Re: Perfomance problem backupscript

[email protected] wrote:


> >> I bet star crawls better than find, btw.
> > GNU find is broken and by default ignores POSIX semantics.
> > This results in much "better performance" than you observe
> > with correctly written programs. Either use sfind or
> > add the -noleaf option to GNUfind.
>
> What ? Even slower ?

??? GNU find usually looks fast because it by default does not stat()
all entries from the directory incorrectly asuming that
the linkcount of a dir is the number of subdirs+2.

Note that this is not even true for traditional UNIX filesystem
in case there are hard linked directories.



> I crawled (still crawl in the stable version) by an old
> quick-hack using popen("ls -a1","r") which i identified
> as malicious memory waster, recently.
> When i employed readdir() instead, i experienced a
> dramatic drop in throughput in overcrowded directories.
> The culprit turned out to be the interchanged calls of
> readdir() and stat(). Some readdir() buffering helped
> much and now it clearly outperforms the ls-hack.
> Nevertheless the performance of ext2 and reiserfs with
> stat() in fat directories is a big disappointment.

Star does slurp in complete directories (when not called
with -lowmem) since more than 3 years. sfind uses exactly
the same module. In future (star-1.6) star and sfind will
share more code because star will use libfind which is made 
from sfind.

> Remarkable: in those filthy directories, find does only
> half as good as ls (which sorts the giant list !).

It is a filthy OS that does not cache decently.
I am not sure if things did change, but 2 years ago, 
the Linux kernel did spend 99% of the time in strcmp()
when working on a large directory.

Compare to Solaris 11.....

Jörg

-- 
 EMail:[email protected] (home) Jörg Schilling D-13353 Berlin
       [email protected]		(uni)  
       [email protected]	(work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily