Re: Perfomance problem backupscript (more details and tests)

Les Mikesell <[email protected]> wrote:

> > I am sorry, but I have the impression that you just don't know POSIX.1-1988 :-(
>
> Just enough to know it didn't match SysV at the time.

Sorry, your statement is not true, please try to check SysV _and_ POSIX.1-1988.


> > The tar archive definition did not change since then, so why should
> > the same thing be useful now but not useful in 1988?
>
> It isn't useful for backups now.  There is no way to specify all files
> that existed at the time an incremental was taken so that others can
> (optionally) be deleted during a restore.  Otherwise you will accumulate
> all files that ever existed when you restore a series of incrementals
> and run out of disk space.  You might as well use cpio.

Are you shure that you know what you are talking about?

First: you are confusing backups in general with "incremental backups".

Second: even "incremental backups" are different from incremental restores.
	Star did support incremental backups quite a long time before it started
	to support incremental restores.

In addition: POSIX.1-1988 tar allows to archive and restore the following
file types: Plain files, hard links, symbolic links, character specials,
block specials, directories, names pipes and contiguous files.

cpio was not in the original POSIX.1-1988 proposal and made it into POSIX
very late under pressure from AT&T. cpio is a very bad archive format.
We should not start to talk about it for this reason.


> > > UNIX?  Is that still around?  I gave up on Solaris when it would have
> > 
> > Are you trolling?
>
> Just enough to offset your hint that Solaris is the One True Way.

The way you use your words rather shows that it is _you_ who is offsetted 
against Solaris without being able to give facts that prove this.


> > In 1993 (the time when Solaris-2.4 was recent), Linux was completely unuseable
> > except fot playing. At that time, Pentium CPUs have been _boringly_ slow....
> > 1994 was the first time, when Intel did come close to the speed of a 1998
> > Sparcstation-II and 1995 was the first time when Intel (with a few too
> > expensive for real use systems) didcome to the same speed as Sparc.
> > PCs of that speed still have been a lot less reliable and even more expensive
> > than Sun systems.
>
> I started at a company that already had the old Sparc, IPX, and IPC
> boxes in 1996.  They were slow, buggy, and crashed regularly, and
> _just_ an OS update would have cost more than faster pentium boxes.

It is obvious that you did never really run such boxes or that you have been
a bad admin. 

Between 1984 and 1993, I did work at a company that did use many PCs _and_ that
was the second biggest OEM for Sun machines (#1 was Kodak) between 1985 and 
1990. We did sell approx. 20% of all Sun 3-50s that have been build.
Even later, I had the fun to compare and before 1996 I would never have even
thought about using a PC instead of a real Sun machine.

Until 1995, PC have been very flaky and did crash frequently while Sun's did
run approx. a year without a reboot.

Regarding speed: do not judge from compile times or from the numbers of 
benchmarks. It is a lot more expensive to compile and optimize code for sparc
processors than doing the same for intel but this does not make sparc slow.

Intel on the other side is a clever company and did influence the C standard
in a way that helped to make benchmarks look nice on Intel based systems. But
there was no relation to reality. If you like to do a real comparison, use 
programs from the real world. I usually use my Reed-Solomon coder from
cdrecord because it does a complex enough task to allow a real comparison.
Run make -f speed.mk in the subdirectory "libedc" in the cdrtools source tree.
Then run e.g. OBJ/i386-sunos5-cc/edcspeed and compare.

Ultrasparc IV systems are interesting here: 

Although they did run at 1200 MHz 2 years ago, they did give similar results
as a Pentium with 2.5 Ghz. The reason is that they have 9 ALUs and many 
registers and they are able to use approx. 7 ALUs concurrently. This gives
interesting results.... If you e.g. check the Reed-Solomon coder from cdrdao
(which is a nearly unoptimized piece of code) you get the same results on
a US-IV (if compiled with a Sun compiler) compared to the cdrtools RS-coder
while the cdrtools Reed Solomon coder runs 2.5 times faster than the one
from cdrdao if you test it on Pentium CPU.

This shows that with real world applications, Sparc is fast enough.....

Also note that Sun Sparc Hardware (if it includes a Sun made MMU) uses a 
special I/O-MMU that speeds up I/O transfers. Sun Hardware is still better 
suited than and Intel or AMD based system if you care about I/O thoughput.


> > Wake up and come back to the real world..... What do you believe where Linux
> > takes nearly all of its new ideas from?
>
> AT&T SysVr4, the same place Solaris got them except for NFS. But AT&T
> didn't know how to price things for the consumer market either.

It seems that you should first try to read a bit about Sysv..... SVr4 
(introduced in 1990 - 15 years ago) of course includes NFS support because it
is build on top of a SunOS-4.x kernel.


> > This incompatible hack was the cause for various incompatibility problems
> > wwith the real world and still is as GNU tar (even the latest version)
> > by default still creates non-POSIX archives.
>
> Like I said, I'd love to see an alternative that can do all the same
> things.

For all my use, I see star as the best way to go. I use it on a dayly base 
since 1984. This is long before GNU tar even exists.....


> > Wrong: star is a drop in replacement for tar.
>
> Star is not a replacement for tar in any usage that is not precisely

WRONG: please first read the man pages for both star and tar and 
try to avoid to again confusing GNU tar with tar.....


> > I am sorry, but do not expect me to implement GNU getopt() bugs just for the
> > fun of being 100% GNU tar compatible. I am using my getargs() which is
> > more flexible that GNU getopt() and even older.
>
> I don't care about command line syntax, except that compatibility would
> help with integration into existing tools.  The showstopper is the
> unnecessary marriage to filesystem boundaries and inability to pick up
> contents of directories renamed across excluded/included areas.

It does not make sense to start implementing the GNU tar bugs just to be
100% compatible with GNU tar.


> > I expect distributions to adopt to the usual conventions instead of
> > suffering from the NIH syndrom like many Linux distributions do.
>
> There are no 'usual' conventions for unix-like systems and never have
> been since the BSD modifications.  As a result, OS distributions have
> to adapt packages to their own conventions.

So you did not get it as you still follow the NIH syndrom....
What you call "adapt packages to their own conventions" is nothing but 
a deviation from useful UNIX rules just because they apply to the NIH
syndrom.

> > OK, but why do you believe do GNU tar maintainers ignore bug reports
> > for serious problems?
>
> Perhaps no one supplies a simple script to reproduce the errors.

I did! But after a few years, I did of course stop wasting my time
with bug reports to the GNU tar maintainers. It makes no sense if
they do not show any reaction on bug reports.


> > GNU is not the center of the workd, if you talk about archive standards,
> > international standard commitees are the right people to look at and not
> > the FSF.
>
> Agreed, in theory, but it doesn't matter in practice.  What matters is
> that you have a program that works.   When the standard committees
> provide reference code or standardize on existing, available reference
> code then everything is fine.  When they make up something that sounds
> nice but doesn't exist, it likely never will and everyone that tries
> will end up with something different.

???

> > Following Open Standards is more important than being open source.....
>
> I'll believe that when I see bug free code being written independently
> to perfectly match these Open Standards.  I don't expect that in
> my lifetime.   Given a choice between using the same code with some
> known defects and using an assortment of different versions that
> almost certainly include new and unknown bugs, I'll usually stick to
> the devil I know.  I'll change my mind when I switch to a platform
> that can't compile C code, I suppose.

I did write star extensions to POSIX in a way that allows them to be integrated
in a future POSIX standard. 


> > I encourage you to edocate yourself about the work of the POSIX standard
> > commitee before you spread FUD on them.....
>
> Have they spec'd a usable incremental backup system?  Has someone
> implemented it exactly to specs?

NO, but star follows all rules for possible extensions that may make it into
a future vaersion of the POSIX standard.


> > > their own maintenance on packages.  However, you are missing the point
> > > here.  I don't want to defend GNUtar, I want to replace it.  I just
> > 
> > You can do it with star.
>
> I can repeat the things that star can't do as often as you can pretend
> that it can...

I don't understand you here: you did already agree that what you like may
be done with the current star if you just change the way you run your
backup. Again: you want a pink pill that cures a disease and I have a 
pill that cures the disease but it is yellow.


> > Star is able to do the backups even for your needs. But please 
> > believe me that I will not implement any hacky new feature that
> > does not even allow to really do more then you could do now,
> > while star is in a state where the current development cycle
> > is in a "die away" state. 
>
> Dump could do my backups if I wanted to follow the same restraints.
> I use tar because I don't want to be restricted to filesystem
> or exclusion boundaries and I don't want the restore operator to
> have to know or care anything about the source topology.  Noting
> new/renamed directories in a way that can traverse devices is
> not 'hacky'.

You again confuse GNU tar with tar. 

	Tar cannot do _any_ kind of incremental backup!

Star can....

Jörg

-- 
 EMail:[email protected] (home) Jörg Schilling D-13353 Berlin
       [email protected]		(uni)  
       [email protected]	(work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily