Re: Re: Perfomance problem backupscript (more details and tests)

On Fri, 2005-10-07 at 10:06, Joerg Schilling wrote:

> > And for incrementals it depends on not crossing
> > filesystem boundaries, unlike any other version of
> > tar, and I think also on not using exclusions or
> 
> I am sorry, but "tar" does not support incremental backups.
> 
> star does and star currently supports backups of filesystems.

If you are going to be this pedantic, please be equally
pedantic in reference to star and mention exactly which
version number has a working incremental mode.

> > Do you
> > have a concise and exact description of the requirements
> > for incremental mode to work properly - including the
> > fact that the restore must also be to a single filesystem?
> 
> ----> man star

Please be more specific.  On one machine 'man star' says nothing
about incremental modes, on another it is there but mentions
nothing about being unable to restore them onto a dissimilar
mount structure (which I'm guessing is the case...).

> > Actually one filesystem I'd like to archive has millions of
> > hardlinks and nothing else I've tried can copy it in
> > a reasonable length of time.  Does star handle the
> > matching efficiently?
> 
> It should do as star (from what I know) is the only tar implementation
> that uses a hashed list instead of a simple linear list.

Thanks - I'll try it when I get a chance.

> > > I never tried to completely understand the GNU tar method
> > > and it is unclear whether this method may work correctly at all.
> >
> > It does work correctly as long as the inode numbers are
> > consistent between runs.  And as you expect with tar,
> 
> I demanstrated yesterday that this is not true....

Sorry, I haven't had time to reconstruct your description. Do you
have a scripted version to demonstrate the issue?  I though we had
this same conversation a year or so ago and the only problem I
could reproduce was if a normal file exists in a full and is
replaced by a directory of the same name by incremental run time.
And I was able to recover the data even in that case.

> And please try not to confuse tar with GNU tar.
> 
> GNU tar is not tar and before GNU tar 1.15.1 was out, GNU tar
> was not even be able to read/write tar compliant archives :-(

I'll make this distinction if you'll start referring to star
by the version descriptions that can restore the incremental
mode archives.  Or will my star that doesn't mention incrementals
in the man page accept the incrementals from the one that does?

> > it does not depend on any particular filesystem boundaries
> > for the run or subsequent restore or any restrictions on
> 
> Please do not confuse a true incremental backup (which is done
> on a whole filesystem and includes _all_ file metadata)
> with a simple tar driectory tree archive.

I think I understand the concept, but only because I sort-of know
how dump does it.  I'm afraid that someone who doesn't know
much about dump would be confused and expect star to be
more tar-like in regard to volumes and mount points.

> > exclusions.  Also, my guess would be that without LVM
> > snapshot support it is more robust on a live filesystem
> > than dump or star's method.  That is, the things that
> > go wrong due to changes during the backup run would be
> > less drastic with gnutar's approach.
> 
> GNU tar could absolutely not be called rubust - it fails miserably
> even with simple backup test cases.

Scripted example, please?


> Before we start to compare star's method with another method,
> please let us first check whether the other method has been implemented
> in a correct way before. GNU tar's incrementals doe not work correctly 
> so it does not make sense to compare them to useable programs
> like ufsdump/ufsrestore or star.

GNUtar does work and has for over a decade.  There may be some
bugs, but it is very usable, and still the only choice if you
intend to restore onto a different partition layout than the
source or modify a directory into a mount point between runs.

Star may very well be better, but it has only been a possible
option for a year or less and I'm not sure the version being
included in popular distributions is the robust one.  And it
requires a different way of thinking about backups than
tar, with it's one-filesystem and whole-filesystem requirement.

-- 
  Les Mikesell
    [email protected]