Re: Perfomance problem backupscript (more details and tests)

On Mon, 2005-10-10 at 16:48, Joerg Schilling wrote:

> Plese tell me where GNU tar should have fewer constraints and please
> prove this.

Perhaps different constraints would be a better description. 

> GNU tar has even more constraints than star or ufsdump.

GNUtar can traverse arbitrary mount points during the backup, will
not fail horribly if you convert what was a directory on a
prior pass into a new mounted filesystem before the next incremental,
and takes no special handling at all to restore on top of
an entirely different mount structure, including doing the right
thing with incrementals.  The restore process does not care about
inodes at all except for reconstructing hardlinks so it is
possible to restore a backup/incremental set onto a VFAT filesystem. 

> star and ufsdump only rely on one single time stamp in /etc/dumpdates
> or /etc/tardumps while GNU tar in relies on a complete list of directories
> and their inode numbers to be retained over the whole incremental backup period.

A tradeoff, since the arbitrary file listing makes it straightforward
to have multiple sets of full/incremental runs outstanding at the same
time.  And with a backup manager like amanda that is all handled
automatically anyway.

> In addition, all these programs rely on st_ctime or st_mtime being newer
> on files that did change in order to make it on the incremental backup.
> 
> Note that if you carefully read this, you will find that GNU tar definitely
> not less fragile than star or ufsdump:
> 
> -	GNU tar depends on the inode numbers of directories and on
> 	ctime/mtime

Yes, but if they don't match, gnutar includes a new copy in the
incremental.

> -	star & ufsdump depend on the inode numbers for all files and
> 	on ctime/mtime. 

I've always imagined that horrible things could happen on an active
filesystem with something that relies on the stability of inode numbers
through the run and Linus has always warned that you should not count
on dump's view of the filesystem to be in sync with anything else under
Linux.  Tar could miss things that are being renamed if something
changes between the directory read and the file open but it won't go
too far wrong.

> As GNU tar does not work _without_ the need to check inode numbers,
> it depends the same way on them as star/ufsdump do.

GNUtar only cares about device/inode numbers on directories and
only checks them against the saved file entry to know if the entire
contents are needed or not (to catch old files under renamed
directories).  It doesn't care about inodes on files at all except
to track hard links on files with more than one name.

>  Star/ufsdump may detect
> _and_ "replay" file renaming if the ctime for the file does not change, 
> GNU tar may detect file renaming but will just remove the old file name
> instead of renaming it during restore.

Yes, to GNU tar a rename translates to a remove and re-create in the
new location.  That makes the re-create part independent of anything
already in the filesystem or not at incremental restore time.  If
you know what is on the tape (as with the indexing option of amanda)
you can pull a file from the incremental run without going back
to the prior full.

> NOTE: you definitely do not need to create the same topology at restore time
> when using star or ufsdump. You only need to restore backups from nested mounts
> _after_ dou did complerely restore the higher level parts of the directory tree.

I don't understand.  If I back up a large combined /, /home, and /var
partition, how would I restore that onto a small / with separate /home
and /var filesystems?  Won't the references to inodes that map renames
in the incrementals mean the wrong thing now that there are 3 device,
inode combinations instead of one unique set?

-- 
  Les Mikesell
     [email protected]