Re: Pipe to tar or other compressor

On Sun 16 Oct 2011 21:50:57 NZDT +1300, Peter wrote:

> How do you pipe to something that will compress well?
> 
> Ive tried this:
> 
>   mysqldump {opts} | 7za a -t7z -an -si -so > outfile.7z
> 
> doesnt work becasue 7z needs to seek
> 
>   mysqldump {opts} | 7za a -txz -an -si -so > outfile.7z

[...]

You seriously need to distinguish between an "archiver" and a
"compressor". An archiver takes multiple files and stuffs them into a
data stream. Examples: tar, and all the mickeymouse stuff you don't want
to have anything to do with, like zip, arj, 7z. A compressor is a
(conceptually simple) filter that applies a transformation to a data
stream to reduce its size without loosing information (or as the jpeg
case may be, with loosing information).

For your convenience, tar (an archiver!!!) allows you to also pipe the
data stream through a compressor or decompressor.

Examples of compressors are compress (you'll have to be old enough to
have used it), gzip, bzip2, lzma. There is a tradeoff between
compression ratio and CPU load, they both go up together. The newer
programs are high in both. lzma blows your mind with its compression
ratio, unfortunately also with its computational requirements. With many
programs you can influence the tradeoff somewhat, and also the memory
requirements (higher memory use yields better compression).

You choose the program/format as per your requirements. mysqldump
produces truckloads of ASCII data in a single stream, so you don't want
an archiver, only a compressor. All of them work well on ASCII data.
lzma might finish tomorrow, the bonus is that there's no data left to
have to deal with...

Volker

-- 
Volker Kuhlmann			is list0570 with the domain in header.
http://volker.dnsalias.net/	Please do not CC list postings to me.

_______________________________________________
NZLUG mailing list [email protected]
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug