Re: The issue of mime parsing

Rickard Armiento <[email protected]>:

> Andrew Rutherford:
>> In the context of a manifest format, could we not send email
>> right now with multiple MIME parts of type message/external-body
>> pointing at different files on the source system?
>
> Yes, this seems to be one way of implementing a "simplified
> MIME"-approach which is very similar to what I described as a
> possible solution in my last reply to Clemens Fischer. However, in
> that reply I did not consider using the MIME access-type to specifiy
> how to obtain the parts since I though that to be derivable from the
> source email adress, but your idea seems clearly better.
>
> Andrew Rutherford:
>> And it doesn't require changing the MIME standard in any way,
>
> I only partially agree here: it requires specifying that only a
> subset of MIME is allowed for im2000 'manifests'. If generic MIME is
> allowed, we would still need to parse it in the im2000 server, which
> was what we started out trying to avoid.

MIME may be enough for hop-by-hop transmissions, but certainly not
for end-to-end use, where recoding _can_ be the substance that make
the communication succeed!  and for hop-by-hop a simple/special MIME
is enough, but think about communication initiated on behalf of a
client between robots.

> Hence, to clarify my notation: we now talk about a 'manifest
> section' whithin a 'notification file' that is an integral part of
> an im2000 email. "To notify a receiver" then equals performing a
> partial email transfer of the notification file.
>
> Andrew Rutherford:
>> My first concern is securing the email on the server to prevent
>> other parties from downloading it.
>
> These problems must be solved for all approaches that use
> traditional protocols for fetching email data. They might be severe
> enough to motivate a dedicated protocol. However, for the issues you
> mention, HTTPS seems to be good enough.

... as the transfer protocol.

> Andrew Rutherford:
>> the part that worries me is how dialup users post messages to their
>> server and specify the deletion policy (after x days, after first
>> retrieval, etc.
>
> The deletion policy could be specified in the notification
> file. But, as for much else in im2000, message upload requires an
> autenticated and encrypted protocol. Maybe this already has been
> discussed in length, but we have a huge number of protocols -- can
> they be combined?
>   i) "IMAP-like" : client access its email folders.
>   ii) "SMTP-like" : sender uploads messages
>   iii) "FTP-like" : receiver downloads message parts.
>   iv) Notfication : senders email server uploads notification file
>                     to receivers mail storage.
>
> It seems to me that all these (including the more exotic features of
> an IMAP-like protocol) could be combined into one autenticated
> "generic network file system" protocol. In case someone out there is
> interested, I would happily pick up this dicussion in a new thread.

this is not an optional feature:  it will be needed.  maybe not by
all participants, but those who need the features it introduces will
not be able to do without.

> Andrew Rutherford:
>> Mentally I'm considering a first cut to be a local-only
>> authenticated SMTP submission system [3] for clients that then
>> gets converted by the server into the new format
>
> As you seem to suggest, a starting point could be to pick up an
> existing implentation of an email server and:
>   i) change the SMTP input code to convert ordinary MIME emails into
>      the here proposed MIME-notification-file + binary parts, and
>      store them so they can be accessed through HTTPS.
>   ii) deliver the notification-file as an ordinary SMTP email.
>      (however, probably it is a good idea to require that the
>      receiving server supports encryted transfers).
>   iii) into the existing 'mailque'-code add an iteration over stored
>      messages, removing messages whoose deletion policy is fulfilled.
>
> If the notification file uses a subset of MIME, and if the receiver
> has a fully featured generic MIME MUA, he should be able to read the
> message without any change of code on his side. However, I'm not too
> happy about this implementation, since it only implements some bare
> basics of im2000.

the following statements pertain to a certain scenario i think im2000
should be able to serve in a sensible manner.

nephew andy wants to send a greeting card to his aunt bertha.  he is
the negligant type, not a computer geek at all, not quite a match for
aunt knowledgable bertha.  she's even into e-government, but this may
(or may not) be off topic here.

so andy doesn't quite know what to send, but worse, all his keys have
expired.  this, he knows, is no problem:  some party will contact him
about a key, or should bertha believe the greeting card is really
From him, this technicality may well go unnoticed.  again  :)

all stations in train are busy, so he has to send the greeting card
using his handy.  still he wants to make it personal, so he selects a
jingle and talks nice words to belauved aunty bertha.

im2000 has to do this:

- note that no valid key is present,
- note that he wants to send a document, but is given an audio stream.

it will contact key servers claiming to carry keys for andy, and
content transformer services to make a document from the audio stream.
also, it will have to offer whatever it can get to bertha and a
certain unknown party only called by three letter abbreveations that
rhyme with EnnEssAa (upocbythlathatrhymewithNSA), which happens to be
interested in all communications bertha is participating in.

bertha sends a notification back requesting signed and encrypted
communication _only_, allowing a grace period of 24 hours.  in the
meantime a service goes online offering audio-2-txt conversions, but
only of unencrypted byte streams and streams encrypted to its
pubkey.  it is highly trusted, even with berthas favourite key
server, and can re-encrypt documents to whichever key is demanded.

andy will have to make a key and get it registered with at least
three signatures at least partially trusted by bertha.  so far he has
transmitted audio to his backing store encrypted by this stores
pubkey only, and later adds his new signature once its available.
with the audio-2-txt converter now known, further notifications are
exchanged, in part automatically on what the exchanging entities
think is on behalf of both andy and bertha.

bertha decides to accept the new key from sloppy andy, because she
thinks its really him and she wants to try the new conversion service
everybody's talking about.  since this particular service has to set
up its reputation, it has so far refused to cooperate with
upocbythlathatrhymewithNSA, as has bertha, but for berthas and andies
needs the communication has progressed as far as they are willing to
trust the various services involved.

result:  upocbythlathatrhymewithNSA doesn't score (this time), bertha
hears a jingle and reads what her nephew sent.

lets face it:  im2000 will not make it into this world if it can't be
used in this scenario!

> Rickard: You are right. However, for an easily parsed notification
> file, could it be okay to combine delivery and storage? I.e.,
> sending a message is then equal to an autenticated upload of the
> notification file.  The server may still have to check that the
> stated sender corresponds to the signature before acting on the
> message, but it would save some protocol chat before the actual
> transfer.

of course this ok!  i just want delivery, storage, key managment,
conversion services and notifications to be named, but abstract
entities accounted for in the protocols to be devised.  if we don't
do this, who's going to want to use it?  im2000 has to offer
something unique that can't be done by anything else easily enough.
this is not the reason i want it to be involved and complicated, but
i want it to handle real needs!

> Rickard: I do not understand the problem with extending the allowed
> subset of MIME (or a new 'manifest' format) to specify that the
> content should be obtained from a streaming source. Can you clarify
> on this?

it's easy to tell participants URLs they can get data from, but the
participants may not be human beeings, and human beeings may not be
able to clearly name how protocol details are named and handled.
there will be automatic communication between servers that come up
and go down while it's going on, and new services will enter the
arena.  andy didn't know up until he met bertha weeks later that she
didn't hear his voice but read it, because he didn't know that her
hearing aid broke!  what he did know, however, was that he had to
supply a key to sign the contents, and besides, he trusted the
services he knew would be handling the transmission and expected
bertha to accept it.

this doesn't mean he wasn't prepared to get a notification of
failure, because this happend often to him in the (then) past, which
he choose to ignore.  and rightfully so!

> Rickard:
>>> [is it] possible to just keep a crypto signature of the manifest
>>> file?
>
> clemens fischer:
>> people might want to fetch a single part of a message, so this
>> single part needs its own checksum, delivery and storage IDs.
>
> True, but does this single part really need its own *signature*?
> What I was trying to say is that I don't think this is nessecary
> if there is a hash for each part inside the notification file,
> and the notification file itself is signed by the sender.

if bertha insisted on both the notification she was presented,
because only the human receiver could finally decide on it, _and_ the
contents to be signed and encrypted, then, i'm afraid, it has to have
its own signature.

if this signature will be revoked three hours later because hos
backing store decided that too many rejections were coming in, that
will be his problem.

Rickard and Andrew, you have been inspiring and you freshed up this
list in a short time!  this is good.

  clemens

PS:  it may go unnoticed, but i thought some MUAs could cause trouble
for people with strict settings on the maximum line width.  emacs
allows me to just hit ESC-q to fill a paragraph, "to make it
pretty".  this is a service nobody asked for, but used to be watched
carefully back in the character terminal age.

consider me a robot obeying a rule.  i transformed all the paragraphs
of this message, including the citations.  this changed the original
formatting, and some may curse me for it, whereas others won't
notice, kinda like im2000 should work (and fail at times).  of course
i hasten to sign this mes as well  :)

-----BEGIN PGP SIGNATURE-----

iD8DBQE+g3XEpdlrZyFBkK8RAl61AJ0dkQyQxhBOPvuFjA5gorvtAGyR4wCfYGd6
4oVFjSruvTL957JlGPoCNvI=
=dDah
-----END PGP SIGNATURE-----