Re: Re: Perfomance problem backupscript

[email protected] wrote:

> > R.J. Baart wrote:
> > now a backup that took in case of 
> > the SuSE distribution 2-3 hours takes now +/- 12 hours.
>
> You should try to find out wether the bottleneck is in
> - filesystem crawling performance

A known problem on Linux

> - star reading, formatting, transfer performance

Did never cause any problem

> - tape drive (and driver) performance

See my mail....


> To find out about the filesystem tree crawler performance
> (which is quite a snail on my system) :
>
>   time find ...your.directories... -ctime 1 >/dev/null
>
> -ctime 1 will force find to actually obtain inode
> information and to not just scan the name lists of
> the directories. No payload data get read, though.
>
> If this is similarly slow as your backup, then the
> filesystem is to blame. (Super large directories ?)
>
> I bet star crawls better than find, btw.

Do not use GNU find for the tests!

GNU find is broken and by default ignores POSIX semantics.
This results in much "better performance" than you observe
with correctly written programs. Either use sfind or
add the -noleaf option to GNUfind.



> To find out wether the tape drive is to blame i would
> pipe the backup into /dev/null rather than into /dev/st0
>
>   time /bin/star ... -acl H=exustar ...your.directories... >/dev/null
>
> (no -f option)
>
> If this is as fast as your old system's backups then
> the tape drive (resp. its new driver) needs mending.

If the tape drive is the same as before, it is obviously the
driver..... See man star  --> -fifostats on how to find
information on the FIFO buffer fill ratio that are a hint
on whether the driver does not accept data fast enough.


> If the first test runs fast and the second one slow,
> then at least a part of the problem occurs during
> the inner operations of star while reading data
> from disk, formatting them and putting out.
>
> One would then have to develop tests which impose the
> same filesystem load but circumvent star in order to
> learn more about payload data handling in your
> filesystem.
> (I do not expect star to be slow in general but rather 
> being hampered by a peculiar situation on your disk.)
>
> A test with -no-fifo would eliminate one more part
> of your usual backup processing chain. -fifo is intended
> to speed up (resp. smoothen) but one never knows ...

On a OS where star is faster when -no-fifo is used, you could
asume that this OS is broken. 

Jörg

-- 
 EMail:[email protected] (home) Jörg Schilling D-13353 Berlin
       [email protected]		(uni)  
       [email protected]	(work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily