Re: Perfomance problem backupscript (more details and tests)

Les Mikesell <[email protected]> wrote:

> On Thu, 2005-10-13 at 07:20, Joerg Schilling wrote:
>
> > Star did support devices sine 1985 (using proprietray enhancements)
> > and PD/SUG tar did start to support POSIX.1-1988 in 1987.
>
> Posix 1.x was less than useful.  It specified nothing that really

I am sorry, but I have the impression that you just don't know POSIX.1-1988 :-(

The tar archive definition did not change since then, so why should
the same thing be useful now but not useful in 1988?


> > Other tar's did start to support POSIX.1-1988 around 1990 (this includes
> > Solaris - the most posular UNIX).
>
> UNIX?  Is that still around?  I gave up on Solaris when it would have

Are you trolling?

> cost more for a software update from the horribly buggy 2.4 to something
> that worked than it did to replace the hardware with faster pentium
> boxes running a free version of Linux.  I've never had a reason to
> reconsider that decision.

You _are_ trolling!

In 1993 (the time when Solaris-2.4 was recent), Linux was completely unuseable
except fot playing. At that time, Pentium CPUs have been _boringly_ slow....
1994 was the first time, when Intel did come close to the speed of a 1998
Sparcstation-II and 1995 was the first time when Intel (with a few too
expensive for real use systems) didcome to the same speed as Sparc.
PCs of that speed still have been a lot less reliable and even more expensive
than Sun systems.

Since 5 years, Solaris-8 is free for personal & research/educational use.
Since half a year, Solaris-10 is free for any kind of use (even commercial).

Wake up and come back to the real world..... What do you believe where Linux
takes nearly all of its new ideas from?


> > GNU tar was created 1989 from PD/SUG-tar by breaking POSIX.1-1988 compliance 
> > of the original source :-(
>
> And thus it became usable.

WRONG :-(

This incompatible hack was the cause for various incompatibility problems
wwith the real world and still is as GNU tar (even the latest version)
by default still creates non-POSIX archives.


> > This is a problem of the obviously missing maintenance for Amanda.
> > As Amanda claims to support ufsdump/ufsrestore, it should take about
> > 2 hours (for a person who knows the Amanda source) to add a different
> > CLI string for a _very_ similar interface.
>
> Agreed - it could now replace dump easily on the backup side. It's not
> a drop-in replacement for tar since it will not work in all the same
> situations.  So you'd need a backup side driver like dump's and a
> restore-side more like tar - not sure where indexing falls.

Wrong: star is a drop in replacement for tar. GNU tar's CLI differs from
tar. While star is compliant to the latest tar standard (to be found in SUSv2),
GNU tar has many compatibility problems: some of them caused by the bugs
in GNU getopt() and others caused by the missing will of the GNU tar maintainers
to let options behave in a compliant way.


> > If you create a link to star that starts with a 'g', the CLI is 99% compatible.
> > Of course star does neither implement the GNU tar incremental CLI
> > nor the strange bugs of the GNU getopt() function.
>
> The 1% that is impossible is the problem.  If it were just syntax we
> could use a shell wrapper.  And neither the GNU or stock getop() works
> with old tar syntax.  That's the one thing that should have been broken
> on purpose by the first usable version and everyone would have forgotten
> it ever existed by now.

I am sorry, but do not expect me to implement GNU getopt() bugs just for the
fun of being 100% GNU tar compatible. I am using my getargs() which is
more flexible that GNU getopt() and even older.


> > BTW: /etc/star.conf would not be compliant to UNIX rules. Star of course
> > reads /etc/default/star and if this is missing or of star is called "tar",
> > /etc/default/tar
>
> There's that UNIX word again.  I expect my software to be fixed to use
> the conventions of my distribution, regardless of the originality of
> the developer.  Packagers for other distributions should do the same.
> I'd prefer that anything built from a developer's tarball land
> entirely under /usr/local, but I don't expect any agreement about that.

I expect distributions to adopt to the usual conventions instead of
suffering from the NIH syndrom like many Linux distributions do.

> > So you don't use GNU tar frequently and don't see the bugs?
>
> Apparently I don't do anything that triggers them.

OK, but why do you believe do GNU tar maintainers ignore bug reports
for serious problems?


> > The first GNU tar bug was a bug I reported in 1993 for GNU tar reading star
> > archives (error message: skipping to next header). This bug has not been
> > fixed until the end of last year. The GNU tar maintainers told me that this
> > was a star bug although it obviously was a GNU tar bug. Last year people
> > did report the same problem to be present with GNU tar archives too.
>
> I have no problem with GNUtar using a unique format.  I don't foresee
> any situation where using GNUtar to extract/restore would present a
> problem, given source availability.   I would be interested in any
> repeatable way to produce GNUtar archives that GNUtar won't read
> itself.

GNU is not the center of the workd, if you talk about archive standards,
international standard commitees are the right people to look at and not
the FSF.

Following Open Standards is more important than being open source.....


> > Now tell me please: do you trust in a tar implementation that is maintained
> > in way that ignores bug repports for showstopper bugs for more than 10 years?
>
> Compatibility with ever-changing standards committee decisions isn't a

I encourage you to edocate yourself about the work of the POSIX standard
commitee before you spread FUD on them.....


> their own maintenance on packages.  However, you are missing the point
> here.  I don't want to defend GNUtar, I want to replace it.  I just

You can do it with star.

> can't until there is another equally filesystem-boundary-agnostic
> replacement.  Star isn't that now, and won't be unless you add a
> similar incremental mechanism.  Even then it would be difficult for
> me to switch without compatibility with my old GNUtar tapes in the
> amanda rotation.  I don't want to lose the ability to tell amanada I
> want a file from some earlier date and have it tell me which tape
> to insert.

This looks like you suffer from a disease and you like to get pink
pills to cure them but I have only blue pills.

Star is able to do the backups even for your needs. But please 
believe me that I will not implement any hacky new feature that
does not even allow to really do more then you could do now,
while star is in a state where the current development cycle
is in a "die away" state. 

The main development objective for 1.5 was incremental backups and
adding cpio archive format support. We are now in a final test state
and in order to avoid doing nothing, I am working on _unrelated_
(and thus simple and separately testable) features that are on the
TODO list for a long time:

-	Better range check for int types.

-	Better security by e.g. setting uid to nobody and clearing
	the TSUID bit in that case.

-	Better warnings.

Also please note that the next star development cycle (for 1.6) 
already has an objective that is important and not handled by GNU tar
at all: Star will start to support path names with unlimited lenght (*).
I believe that this is more important than implementing just a different
backup method with no really new features.

*) Well the path length limit will be close to 8 billion characters.


As a result of the way this will be implemented, there will be a lot
exciting new features that are completely new to any known tar implementation.

And _before_ I consider to implement a different backup method 
that does not really give new features, I will need to add support
for extended attribute files in order to support all Solaris features.

Jörg

-- 
 EMail:[email protected] (home) Jörg Schilling D-13353 Berlin
       [email protected]		(uni)  
       [email protected]	(work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily