Re: Perfomance problem backupscript (more details and tests)

Les Mikesell <[email protected]> wrote:

> Perhaps different constraints would be a better description. 

I don't see any proof that the constraints are different.
GNU tar has the same constraints - they only apply to different parts
of the backup, but this does not make GNU tar more tolerant against
the already discussed problems.


> > GNU tar has even more constraints than star or ufsdump.
>
> GNUtar can traverse arbitrary mount points during the backup, will
> not fail horribly if you convert what was a directory on a
> prior pass into a new mounted filesystem before the next incremental,
> and takes no special handling at all to restore on top of
> an entirely different mount structure, including doing the right
> thing with incrementals.  The restore process does not care about
> inodes at all except for reconstructing hardlinks so it is
> possible to restore a backup/incremental set onto a VFAT filesystem. 

If I do a stat(2) on a vfat filesystem on Solaris, I see a number in st_ino and 
for this reason, I see no problem to extract a backup on a VFAT system unless
the source filesystem includes hard linked files. But GNU tar will eually fail
in this case.


> > star and ufsdump only rely on one single time stamp in /etc/dumpdates
> > or /etc/tardumps while GNU tar in relies on a complete list of directories
> > and their inode numbers to be retained over the whole incremental backup period.
>
> A tradeoff, since the arbitrary file listing makes it straightforward
> to have multiple sets of full/incremental runs outstanding at the same
> time.  And with a backup manager like amanda that is all handled
> automatically anyway.

Could you please read the man page of star before you reply the next time?

You may of course tell star the name of the dumpdates file...


> > In addition, all these programs rely on st_ctime or st_mtime being newer
> > on files that did change in order to make it on the incremental backup.
> > 
> > Note that if you carefully read this, you will find that GNU tar definitely
> > not less fragile than star or ufsdump:
> > 
> > -	GNU tar depends on the inode numbers of directories and on
> > 	ctime/mtime
>
> Yes, but if they don't match, gnutar includes a new copy in the
> incremental.

And whay do you believe that this should help?
It has absolutely no advantage. Please first try to understand how the
star incremental restore works.


> > -	star & ufsdump depend on the inode numbers for all files and
> > 	on ctime/mtime. 
>
> I've always imagined that horrible things could happen on an active
> filesystem with something that relies on the stability of inode numbers
> through the run and Linus has always warned that you should not count
> on dump's view of the filesystem to be in sync with anything else under
> Linux.  Tar could miss things that are being renamed if something
> changes between the directory read and the file open but it won't go
> too far wrong.

Could you tell us why you write this in a discussion where you like to
convince us why GNU tar includes features that star is missing?
Ufsdump is extremely sensitive against active filesystems because it first
reads the inode array _before_ starting to archive directories and then
file contents.
Star and GNU tar are less sensitive because they only depend on the 
set of names in a directory not to shrink during the time this dir
is archived. 

However, if you like to make a reliable backup from a life machine,
you need to use a filesystem snapshot anyway.

>
> > As GNU tar does not work _without_ the need to check inode numbers,
> > it depends the same way on them as star/ufsdump do.
>
> GNUtar only cares about device/inode numbers on directories and
> only checks them against the saved file entry to know if the entire
> contents are needed or not (to catch old files under renamed
> directories).  It doesn't care about inodes on files at all except
> to track hard links on files with more than one name.

So you like to use the statistical probbability as a gauge?


> > NOTE: you definitely do not need to create the same topology at restore time
> > when using star or ufsdump. You only need to restore backups from nested mounts
> > _after_ dou did complerely restore the higher level parts of the directory tree.
>
> I don't understand.  If I back up a large combined /, /home, and /var
> partition, how would I restore that onto a small / with separate /home
> and /var filesystems?  Won't the references to inodes that map renames
> in the incrementals mean the wrong thing now that there are 3 device,
> inode combinations instead of one unique set?
>

The way I did write yesterday, please read the related mail, it contains
the algorithm.

Jörg

-- 
 EMail:[email protected] (home) Jörg Schilling D-13353 Berlin
       [email protected]		(uni)  
       [email protected]	(work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily