Re: Perfomance problem backupscript (more details and tests)

Les Mikesell <[email protected]> wrote:

> > First: you are confusing backups in general with "incremental backups".
>
> Backups without incrementals won't work for me because they won't
> fit on the tapes.  

Please try to be exact enough to have a discussion without miss understandings.


> > Second: even "incremental backups" are different from incremental restores.
> > 	Star did support incremental backups quite a long time before it started
> > 	to support incremental restores.
>
> 'Support' for incrementals should mean that it includes old files under
> renamed directories during an incremental regardless of the location
> during the full run.  Star can miss them if you move out of an excluded
> area.

This is your private opinion but not necessarily an objective statement.
Every tool may not work as expected if used in an undomumented way.
Star's incrementals work perfectly as long as you follow the documentation.


> Restoring incrementals must have the option to delete files that were
> not present at the time of the incremental run or you will accumulate
> all the garbage that was deleted between the full and incrementals and
> probably run out of space.

It is really boring to see that you still did not read the star man page....
This is not the first time that you write untrue things because you don't
read the man page.

The next time you will write things that may be proven wrong by reading
the star ducumentation, I will consider this thread to be useless and
stop it!

> > > Just enough to offset your hint that Solaris is the One True Way.
> > 
> > The way you use your words rather shows that it is _you_ who is offsetted 
> > against Solaris without being able to give facts that prove this.
>
> All I can do is describe my experience. Admittedly with old hardware
> and old/buggy software.

Then please admit that your experiences are a result of badly administred
or defective and unfixed machines. I have enough experiences with Sun
equipment to prove that your experiences do not match reality.



> > > I started at a company that already had the old Sparc, IPX, and IPC
> > > boxes in 1996.  They were slow, buggy, and crashed regularly, and
> > > _just_ an OS update would have cost more than faster pentium boxes.
> > 
> > It is obvious that you did never really run such boxes or that you have been
> > a bad admin. 
>
> I didn't install or configure them.  I just restarted them when people
> said the services weren't working and put up with sales calls about
> upgrades we couldn't afford.

See above: Every computer that is badlly administered or defective may cause
problems. Your experiences do not match reality.


> > Between 1984 and 1993, I did work at a company that did use many PCs _and_ that
> > was the second biggest OEM for Sun machines (#1 was Kodak) between 1985 and 
> > 1990. We did sell approx. 20% of all Sun 3-50s that have been build.
> > Even later, I had the fun to compare and before 1996 I would never have even
> > thought about using a PC instead of a real Sun machine.
>
> This was 1996.  Admittedly I was replacing old Suns with new PC's, but I
> came out ahead and except for a few people who used those big
> non-multisync monitors as space heaters all anyone saw was a speedup.

If you in 1996 did run a system with a OS from 1993, it is obvious that you have
not really been interested in that machine. Did you do similar things with
machines running Linux at that time? I am sure that not.



> > Until 1995, PC have been very flaky and did crash frequently while Sun's did
> > run approx. a year without a reboot.
>
> Several of these Sun's crashed or rebooted at least monthly.  They
> didn't have a lot of RAM and I suspected a memory leak in the OS.
> I've forgotten what version of sendmail was on them but it was far
> behind what Linux distributions had at the time and was known to be
> buggy.

I am sorry, but if your sysadmin was not doing a good job, please don't
tell us that Solaris was bad - tell us that you would have needed a different
sysadmin but not a different OS.


> > Regarding speed: do not judge from compile times or from the numbers of 
> > benchmarks. It is a lot more expensive to compile and optimize code for sparc
> > processors than doing the same for intel but this does not make sparc slow.
>
> You need to compare dollars to dollars not just processor to processor.

Even then, Sun systems did win in 1993 (and you did write about systems from 
1993!).


> > It seems that you should first try to read a bit about Sysv..... SVr4 
> > (introduced in 1990 - 15 years ago) of course includes NFS support because it
> > is build on top of a SunOS-4.x kernel.
>
> Which came from AT&T SysVr2 with TCP and utilities from BSD bolted on.
> All of the things we think of as 'like unix' were already in SysVr2,
> although the OSI-based networking was a little different and RFS was
> very different from NFS.

??? this looks confused. What do you like to say?


...
> tape to insert.  I'm not interested in switching to a system that
> does not offer these features.  The current star might do the basics,
> but I'm not sure how amanda/star would deal with indexing such that
> it knows what tapes you need and in which order to restore an
> old file that was under a directory renamed before an incremental.

Star does meet your criteria, it seems that amanda is currently not well 
maintained.

> > > > Wrong: star is a drop in replacement for tar.
> > >
> > > Star is not a replacement for tar in any usage that is not precisely
> > 
> > WRONG: please first read the man pages for both star and tar and 
> > try to avoid to again confusing GNU tar with tar.....
>
> Perhaps you are confusing my system with Solaris. On my GNU/Linux system
> tar is GNU tar and there is no proprietary or BSD version. 

Linux does not have tar at all, Linux is a kernel only.

Most Linux based OS distributions (like e.g. SuSE) come with a program named 
"tar" that is definitely not tar. Prove by this facts: until recently, this 
program was neither able to read or write standard compliant tar archives 
correctly. This program does not meet the SUSv2 standard tor the "tar command
line interface". This program still does not write standard compliant tar by 
default. 

Please note that "star" does meet all these criteria if called "tar".
Also note that the program "tar" on Solaris also meets the above criteria.


> > It does not make sense to start implementing the GNU tar bugs just to be
> > 100% compatible with GNU tar.
>
> Do it any way you like, as long as you can match the feature of
> including old files under renamed directories regardless of their
> original location.  If they exist in the included part of the
> incremental run, they must exist after a restore.  I don't see
> any way to claim that omitting them is correct under any circumstances.

Star already implements _working_ incrementals. GNU tar incrementals
still do not always work correctly.

Please do not ask me to implement _another_ and _different_ method for
incrementals as long as there are many other and more important things on
my TODO list for star.

Note that star _is_ already able to make incremental backups that fit
your needs using the _currently_ implemented algorith. You just need
to use star the right way...


> > > > OK, but why do you believe do GNU tar maintainers ignore bug reports
> > > > for serious problems?
> > >
> > > Perhaps no one supplies a simple script to reproduce the errors.
> > 
> > I did! But after a few years, I did of course stop wasting my time
> > with bug reports to the GNU tar maintainers. It makes no sense if
> > they do not show any reaction on bug reports.
>
> Well, now you have a second chance since the Linux distribution
> maintainers fix bugs too.   Post it to the RedHat or fedora bugzilla
> and let them fix it or push it upstream.  Nobody really like the days
> when you had to build your own distribution from source but times have
> changed and you can take advantage of the help.

I did explaint it already several time to you:

Redhat maintainers did definitely not fix bugs in GNU tar, they did just
publish otherwise unpublished internal test versions from the real GNU tar 
maintainers.

And to give you a hint on my experiences with Debian, Redhat and Suse
binary packets from my programs star & cdrtools: Typical maintenance of 
programs looks like the "maintainers" just bastardize my programs and the
compile results from original sources from did always work better than the 
binary versions supplied with Linux based OS distributions.


> This is where you were supposed to point out the value of the open spec
> by listing all of the correctly interoperating versions that have been
> developed following the spec - and how I don't have to worry about any
> differences among them because the spec covers everything needed for
> incremental backups to always work correctly even when the code base
> for the backup tool varies wildly from that used to restore.

The POSIX specs currently don't say anything about backups.

> > I don't understand you here: you did already agree that what you like may
> > be done with the current star if you just change the way you run your
> > backup. Again: you want a pink pill that cures a disease and I have a 
> > pill that cures the disease but it is yellow.
>
> If star 'just worked' with amanda, I might be able to switch because
> my existing backups happen to be matched to filesystem boundaries
> now (but I'd have to check the exclusions).   I'm also somewhat
> interested in using it as the transport for backuppc if it would
> get acls right.  Backuppc currently only uses the timestamp for
> tar incremental operations, so only the rsync transport is exact.

If you have problems with amanda, please complain to the amanda maintainers!

Star tries to give you a rock solid method for incremental backup and restore.

If you don't like the interface that star offers to impement this feature,
it seems to be natural to find high level programs that give you features
that base on the features from star but provide a different way to use.

Jörg

-- 
 EMail:[email protected] (home) Jörg Schilling D-13353 Berlin
       [email protected]		(uni)  
       [email protected]	(work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily