Re: Perfomance problem backupscript (more details and tests)

On Wed, 2005-10-19 at 06:24, Joerg Schilling wrote:

> > > First: you are confusing backups in general with "incremental backups".
> >
> > Backups without incrementals won't work for me because they won't
> > fit on the tapes.  
> 
> Please try to be exact enough to have a discussion without miss understandings.

There are a lot of impractical ways to do daily backups. If I have to
choose one of them I'll probably use disk imaging. For the purpose of
this discussion I am only concerned with practical ways, which for
me, requires correct and complete incrementals.

> > 'Support' for incrementals should mean that it includes old files under
> > renamed directories during an incremental regardless of the location
> > during the full run.  Star can miss them if you move out of an excluded
> > area.
> 
> This is your private opinion but not necessarily an objective statement.
> Every tool may not work as expected if used in an undomumented way.
> Star's incrementals work perfectly as long as you follow the documentation.

My opinion is that an incremental run should include all files that
exist in the included area.  As a practical matter, I need to
exclude some areas and I cannot restrict renaming.  The tool I use
must work correctly under these conditions.

> > Restoring incrementals must have the option to delete files that were
> > not present at the time of the incremental run or you will accumulate
> > all the garbage that was deleted between the full and incrementals and
> > probably run out of space.
> 
> It is really boring to see that you still did not read the star man page....
> This is not the first time that you write untrue things because you don't
> read the man page.

There is nothing untrue or uniquely related to star in my statement
above.  It is a basic requirement that star may recently be able to
meet.  It is ignored in the Posix spec, as far as I can tell, though.

> > All I can do is describe my experience. Admittedly with old hardware
> > and old/buggy software.
> 
> Then please admit that your experiences are a result of badly administred
> or defective and unfixed machines. I have enough experiences with Sun
> equipment to prove that your experiences do not match reality.

Yes, running the buggy Solaris 2.4 was bad administration. But that's
what the Sun people sold us. I'm sure you stopped using it at the
first opportunity.

> If you in 1996 did run a system with a OS from 1993, it is obvious that you have
> not really been interested in that machine.

The only way to get a working OS from Sun at the time was to buy a new
one to replace it - for more than faster hardware with a free OS would
cost.  When someone sells me a problem I'm not thrilled about having
to pay extra to fix it and it certainly didn't make sense to pay
more than a faster working system would cost. 

>  Did you do similar things with
> machines running Linux at that time? I am sure that not.

I always applied the updates that were freely available.

> > Several of these Sun's crashed or rebooted at least monthly.  They
> > didn't have a lot of RAM and I suspected a memory leak in the OS.
> > I've forgotten what version of sendmail was on them but it was far
> > behind what Linux distributions had at the time and was known to be
> > buggy.
> 
> I am sorry, but if your sysadmin was not doing a good job, please don't
> tell us that Solaris was bad - tell us that you would have needed a different
> sysadmin but not a different OS.

The systems were running stock Sun software, and that is what was
crashing.  I think Sun has become more sensible about providing
software updates, but it's too late for me.

> > > It seems that you should first try to read a bit about Sysv..... SVr4 
> > > (introduced in 1990 - 15 years ago) of course includes NFS support because it
> > > is build on top of a SunOS-4.x kernel.
> >
> > Which came from AT&T SysVr2 with TCP and utilities from BSD bolted on.
> > All of the things we think of as 'like unix' were already in SysVr2,
> > although the OSI-based networking was a little different and RFS was
> > very different from NFS.
> 
> ??? this looks confused. What do you like to say?

If you are going to pretend that there is a standard for UNIX, you have
to go back to the AT&T versions before all of the variations were
added.  Since that time there has been no standard and every vendor
has gone its own way.  Then it follows that applications have to
be adapted to match each distribution's conventions.

> ...
> > tape to insert.  I'm not interested in switching to a system that
> > does not offer these features.  The current star might do the basics,
> > but I'm not sure how amanda/star would deal with indexing such that
> > it knows what tapes you need and in which order to restore an
> > old file that was under a directory renamed before an incremental.
> 
> Star does meet your criteria, it seems that amanda is currently not well 
> maintained.

I was hoping you would fill in the details here about what amanda would
have to do to know the tapes needed in this circumstance. It has been
so long since I used dump under amanda that I don't remember how it
worked there.  I think it may have dropped you into dump's interactive
mode and fed it the whole set of levels, traversing them once for
the index and again for recovery after making selections. With GNUtar
based runs, the data is always included in the same run as the name so
a restore of renamed files or paths is simple.

> > Well, now you have a second chance since the Linux distribution
> > maintainers fix bugs too.   Post it to the RedHat or fedora bugzilla
> > and let them fix it or push it upstream.  Nobody really like the days
> > when you had to build your own distribution from source but times have
> > changed and you can take advantage of the help.
> 
> I did explaint it already several time to you:
> 
> Redhat maintainers did definitely not fix bugs in GNU tar, they did just
> publish otherwise unpublished internal test versions from the real GNU tar 
> maintainers.

That's easy enough to disprove.  Their published source RPMs contain
the pristine developers tarball plus separate patches for the
distribution and a spec files that applies them while the binary
is built.  I've already mentioned that the current source RPM
contains seven patches - one for a specific bug you mentioned
earlier.  They also keep a bugzilla of open bug reports.  Should I
expect to see yours there?

> And to give you a hint on my experiences with Debian, Redhat and Suse
> binary packets from my programs star & cdrtools: Typical maintenance of 
> programs looks like the "maintainers" just bastardize my programs and the
> compile results from original sources from did always work better than the 
> binary versions supplied with Linux based OS distributions.

And I'll report the experience with cdrecord from the user side.  It has
always worked on the RedHat versions I've used.  The only quirks are
whether or not a particular version defaults to scsi emulation with
confusing device naming or uses more transparent driver access to IDE
devices.  

> > This is where you were supposed to point out the value of the open spec
> > by listing all of the correctly interoperating versions that have been
> > developed following the spec - and how I don't have to worry about any
> > differences among them because the spec covers everything needed for
> > incremental backups to always work correctly even when the code base
> > for the backup tool varies wildly from that used to restore.
> 
> The POSIX specs currently don't say anything about backups.

Then perhaps you might understand why they are considered irrelevant
by a system administrator.  Why do you keep bringing them up?

> Star tries to give you a rock solid method for incremental backup and restore.

If you don't exclude anything - or don't rename any directories - or
some unlucky combinations of those things.

-- 
   Les Mikesell
     [email protected]