encoding issues, was: What I see as problems to solve ... and a strawman solution

On 1-feb-04, at 21:48, Kai Henningsen wrote:

> This is strictly aimed at headers (I certainly don't see it as useful 
> to
> put the bodies in there), and my argument is that XML is *vastly* 
> simpler
> than what we have today with mail headers. Which is a rather
> uncomplimentary thing to say about current mail headers.

> As far as I can make out, the RFC-covered area currently has three
> (entirely) different "general" technologies to describe small amounts 
> of
> heterogenous data: the RFC-822 family, XML, and ASN.1 (for example in
> SNMP).

These are all difficult to parse. (Obviously the right library makes 
all your troubles disappear along with most of your performance.)

So what does this tell us? Are general purpose encoding mechanisms by 
definition difficult to parse?

My main issue is with efficient transport. Obviously any kind of 
escaping of special values is deadly here, so we must go binary for any 
and all content. But what about the headers? As the word itself already 
indicates, these go at the beginning of the message. That's not so 
great, as this makes it necessary to expand the message every time it 
passes an intermediate hop. This may not always be bad, but it often 
is. I would be much happier if the transit headers (as opposed to the 
ones that are only of interest to the recipient) were kept apart from 
the content. And if the content and headers are stored in a single 
file, the headers should go at the end of the file. Note that there are 
several file formats that allow different chunks of content to live 
together in the same file, such as the IFF format and its derivatives. 
(Think WAV, AIFF, AVI.)