Re: Re: Perfomance problem backupscript (more details and tests)

On Sun, 2005-10-09 at 07:39, Joerg Schilling wrote:
> >
> > If you are going to be this pedantic, please be equally
> > pedantic in reference to star and mention exactly which
> > version number has a working incremental mode.
> 
> Trying to be exact is different from being pedantic.

I'm only interested in being as exact as necessary. Almost
any distribution of an OS I'm likely to use will have a
program named tar that will extract any of my archives
because it is in fact gnutar and it will do it regardless
of the destination filesytem type, even to VFAT, and
regardles of any intervening mount points.  Working
with star, I would have to be much more precise about
which version is available if the distribution has
it it at all.  

> What people understand by "tar" is what appeared on UNIX around 1977 and
> what is SUSv2 specifies. Check out both, the SUSv2 standard and any UNIX
> variant and you will find out what "tar" really supports.

I understand the difference but the last time I used the old
version of tar was on a freebsd 4.x long ago and it was
unable to handle the 32-bit device numbers of that distribution.
I don't ever care to see that version again.  And, I'll
venture a guess that an order of magnitude more people have
gnutar as their one and only tar program than anything else.

> As you should know, I am always referring to the most recent star
> version. If you like to know when a specific feature has been 
> introduced, please check the change logs....

I and just about everyone else have come to depend on packaged
programs bundled into distributions that are tested and have
regular updates.  I might look at the change logs if they
come along with the update, but what I really want to know
is whether I can count on the star included with any particular
distribution to interoperate with the one in some other
particular distribution.

> > Please be more specific.  On one machine 'man star' says nothing
> > about incremental modes, on another it is there but mentions
> > nothing about being unable to restore them onto a dissimilar
> > mount structure (which I'm guessing is the case...).
> 
> I am sorry, but if you did not install a recent star on your machine
> this is your fault.

Does any version of star note the requirements for restoring? 
I'm assuming a posix-like filesystem is needed and no mount
points in the tree where you want the files restored, but
that has not been made clear anywhere that I can find.


> > > > Actually one filesystem I'd like to archive has millions of
> > > > hardlinks and nothing else I've tried can copy it in
> > > > a reasonable length of time.  Does star handle the
> > > > matching efficiently?
> > > 
> > > It should do as star (from what I know) is the only tar implementation
> > > that uses a hashed list instead of a simple linear list.
> >
> > Thanks - I'll try it when I get a chance.
> 
> Well, star did introduce this 8 years ago, it is one of many reasons
> why star is much faster than other tar implementations.

Does it work in -copy mode?  I started a copy of about 100 gigs
on Friday afternoon and it isn't finished Sunday evening. There
are many errors on the screen like:

star: pc/dns1/833/f%2f/fetc/fbigstart/freinit/fS30httpd: Linkcount below
zero (-4)
star: pc/dns1/833/f%2f/fetc/fbigstart/freinit/fS10sshd: Linkcount below
zero (-5)

when in fact the source link counts are:
-rw-r-----  71 backuppc backuppc  29 Aug 11  2004 fS30httpd
-rw-r-----  71 backuppc backuppc  28 Aug 11  2004 fS10sshd

The filesystem is active and the counts could have changed
by a few during the run, but not enough to go to zero or
negative.


> > I'll make this distinction if you'll start referring to star
> > by the version descriptions that can restore the incremental
> > mode archives.  Or will my star that doesn't mention incrementals
> > in the man page accept the incrementals from the one that does?
> 
> ???
> 
> I am always as exact as possible. If you install/use an outdated version
> of star, it's your fault.

What I use is almost always the expert choice of the person
packaging a program for a distribution.  That means it won't
be what some developer added as an experiment last week, but
rather when the documentation makes a convincing argument that
the new version is better and it has been though some testing.

> > I think I understand the concept, but only because I sort-of know
> > how dump does it.  I'm afraid that someone who doesn't know
> > much about dump would be confused and expect star to be
> > more tar-like in regard to volumes and mount points.
> 
> This would only apply to people who don't read manual pages - sorry.

That's most of them, but even so I don't agree.

> As you cannot use the incremental features without understanding them
> (which means reading the man page), people who use the feature should
> know how it works.

They have to know quite a lot about filesystems to figure that
out.  I doubt that anyone not already familiar with dump would
figure out the requirements for restoring incrementals.


> > Star may very well be better, but it has only been a possible
> > option for a year or less and I'm not sure the version being
> > included in popular distributions is the robust one.  And it
> > requires a different way of thinking about backups than
> > tar, with it's one-filesystem and whole-filesystem requirement.
> 
> If you run a OS distribution that does not check for newer versions
> and that does not update programs if apropriate, you either know this
> and do it yourself or you need to avoid this distribution because it is 
> not doing a good job.

The distributions I use provide updates, but only bug and security
fixes, not behavior-changing modifications.  The lastest version
I can find included in fedora FC4 is 
# rpm -q star
star-1.5a54-2
but that's a bleeding-edge distribution whose purpose is to
get the first large scale testing on new packages.  I run many
more machines on Centos3 which has a more conservative
star-1.5a08-4 package.

-- 
  Les Mikesell
    [email protected]