Re: Replacing SMTP Et Al

On Fri, Jan 30, 2004 at 05:06:26AM -0000, James Craig Burley wrote:

> I can't remember who said this, but it has been said many times: it
> may be pointless to create and deploy an NG email system, as long as
> clients have the option of using SMTP.

And before RFC821 got widely accepted, UUCP bangpaths were seen as the way 
to do things. The great thing about SMTP is that the address format became 
easier to handle - it addressed one of the biggest problems with UUCP mail.

A NG mail system should have several objectives:

- A realisation that it is not Next Generation. It is an evolution we hope 
to present as the current generation. Putting "Next Generation" into the 
name makes it look rubbish a decade after it's been deployed. Think Star 
Trek here. :-)

- It should address the biggest problems perceived with current protocols 
without losing the things we like about the current protocols.

- To reduce complexity, it may be advisable to look at extending the 
existing protocol unless the bad things about the current protocol are so 
terribly awful, there is no choice but to remove it.

These should, in my opinion, become our mantras.
 
> As sites deploy NG email to try it out, they will, at some point, come
> to appreciate the ability to exercise more-fine-grained control over
> delivery of envelopes vs. messages (header+body), enjoy better overall
> performance, and so on.

Is performance an issue? If anything, mail systems are too efficient - 
that's why it is so cheap and cost effective to send spam, probably the 
biggest problem with SMTP. Please note, I am not advocating raising the 
price of sending e-mail, or slowing it down. I come to this with the 
viewpoint that authentication, authorisation, in-built whitelisting and 
other techniques to reduce spam that do not require bayesian filtering and 
constant battling with spammers, are the way forward.

Unlike certain people, I do not believe the way forward is to build 
transactional charging into the e-mail system.
 
> Meanwhile, they'll still be dealing with all the spam and vermin they
> get via SMTP.  That won't likely go away.

Indeed. In fact, despite bayesian filtering, the constant addressing of open 
relaying, the real-time blackholing we use today, most people would agree 
spam is on the increase.
 
> As more of their *valid* clients switch sending via NG email, they'll
> utilize the fine-grained controls to deal more effectively with spam
> and vermin that adapts to use NG email instead of SMTP...

Well, the trick is, I think, to make sure that the spammers are going to
find it difficult to switch - we already have laws preventing spam in most
juridstictions, but the problem at the moment is that it is difficult to
actually bring a prosecution because of the nature of how the spam is sent.  
Once you ensure that spammers find it hard to evade either law enforcement,
or their mail not getting where they want it to, the motivation for valid
users to switch becomes huge. The trade-off here is privacy. At one level we 
want accountability, and at another level we need to make sure those who 
wish to protect their identity are able to do so securely. This sounds like 
a contradiction, but I think it can be done.

If there is better handling of privacy and encryption, well, you're starting
to provide real incentives for this to progress. If I actually AM able to
send you my credit card number via e-mail securely, and the tools do not
require the skills of an undergrad in CompSci to use... well, I think we can
see the benefits that kind of technology could bring.
 
> ...because SMTP can and will be made *artificially* more expensive via
> tactics such as Greylisting, Defer Hostility, Tarpitting, and so on.

These are not bad things in themselves. The problem is, they are being added 
onto a protocol that fundamentally does not care about authenticating the 
senders of messages and being able to give control to recipients over who 
they receive e-mail from. The tools can be added to SMTP now, but the 
complexity of doing so is high.
 
> (Note that Dan Bernstein's QMTP is probably very narrowly deployed,
> yet, apparently, there are spamware/verminware exploiters of it!)

An extra 1% of hits doesn't sound much, but when you're dealing with 250 
million e-mail addresses, that 2.5 million extra can make a difference...
 
> As an example, consider an email that is coming from a
> previously-unknown source, but doesn't look all that suspicious to a
> server receiving it.
> 
> With NG email (as I conceive it anyway ;-), the server can indicate
> that it can and will deliver the envelope, but that it won't take
> responsibility for the message, yet it "will call" for it later on if
> desired.

That is one approach, but not one I would agree would actually defeat
spammers or viruses or malware in itself. I think it would just push the
marketing into the envelope of whatever system you design around that 
approach - MAIL FROM: Hot Babes at xxx.com <[email protected]>
 
> For now, I think we shouldn't worry about this too much, and focus on
> just designing the best, cleanest, simplist email system we can, based
> on our understanding of today's system, which, after all, was just a
> prototype, right?

Well, yes that makes sense. I think the best thing to do right now though is
to discuss how we go about this. We all have our own ideas, and it might
take some time for each of us to pick over them all. How about we invite
everybody to submit a post on their ideal NG mail system. We throw them all
into the pot and the threads that follow their posting here discuss the
pluses and minsues. Should kick-start the discussion nicely, I think and 
lets us address some big radical ideas instead of picking over smaller 
points like we've done here.

Sound reasonable to everybody?

-- 
Paul Robinson