Re: gzip/deflate compression/encoding

On Jun 29 2005, Bruce Lilly wrote:

> MIME transfer encoding is typically used end-to-end.  It doesn't
> matter where the bottleneck is, doesn't require any special software
> support at intermediate sites.

Yes, this is the crucial point. You've identified one problem at intermediate
nodes of the network, namely the SMTP size limit which depends on the
installed MTA software. 

With CTE: No software change at intermediate nodes is required. 
 Major change at all client leaf nodes is required just to continue
using email, because the CTE makes the compressed content opaque.

Without CTE, but TLS: changes at intermediate nodes and leaf nodes 
bring definite benefits, but no change is required of all clients at once.

And finally: if the user sends precompressed documents as 8bit attachments 
from his MUA, no changes at all are required today. This is the purest
end-to-end scenario.

>  
> > > See RFC 1123 section 5.3.8.
> > 
> > Seems a little outdated ;-) 
> 
> Limits still exist. Now there's an extension so that the client can
> specify the size before sending.  If the SMTP receiver can't handle
> the message size, the options are compress, fragment, or bounce.  A
> bounce is a failure, fragmentation has several issues, and that
> leaves compression.

Indeed, compression is always an option whether or not a CTE is available.
Just because the sender pushes an uncompressed message over the wire doesn't
mean that the receiver must accept the uncompressed stream into a holding
buffer the same size as the message, it can be compressed as it is received. 

If you're going to give the SMTP receiver an option to rewrite the
message encoding with a CTE, then on-the-fly compression in RAM or on
disk is also an option which compares favourably.

> 
> > Even if you're transferring say an MPEG video stream,
> > there are key frames which allow small random corruptions to be ignored.
> > But compress the video, and you can throw it away if there's a corruption. 
> 
> MPEG *IS* compressed!  Loss of a P, B, or I frame will be a (temporary)

Sorry, I was aware of that, I meant a second CTE compression on top
of the video stream. Regarding text compression, big savings are possible
but normally text isn't the bulkiest part of a large email, so compression is 
not crucial.

> Conclusions:
> 1. binary-to-8bit encoding w/o gzip compression is fine for audio,
>    image, and video media which already has media-specific compression
>    (for 8bit transport, avoiding the 37+% expansion of base64)

I agree with Frank Ellermann, this option isn't broken and has the advantage
of simplicity.

> 
> No, the information is all there (both formats can be opened in OpenOffice
> and contain the same spreadsheet data).

Except for VB macros and whatever else Microsoft decided to put in
their files. I've been told that Excel is Turing complete...

> > Of course, all these transparent ways must be implemented and deployed
> > just the same, but what is the serious long term benefit in making MIME
> > carry CTEs? Every MIME aware application will have to be adapted to
> > read those new encodings.
> 
> See Harald's message.

I really don't know, it seems easy to define these CTEs, but once they exist
everyone is stuck with them and there's no backwards compatibility.

If it really has to be tried out, wouldn't a few content types
make more sense to test the waters? A CTE which replaces them can
always be defined some years later, and in the meantime it doesn't
force all standards compliant software to implement a decompressor.

-- 
Laird Breyer.