Re: Comments on IM2000

>It was thus said that the Great James Craig Burley once stated:
>> 
>> Under my proposal, there's no question the "average" user would have
>> little trouble sending hundreds, even thousands, of pieces of outgoing
>> UBE without running into resource limits.  But spammers won't find
>> email to be such a worthwhile protocol to advertise their goods at
>> those rates of usage, especially given an increasingly sophisticated
>> end-user audience (and more-intelligent MUAs).
>
>  What about mailing lists?  Say, a real popular one, like the Linux Kernel
>Mailing list?

It's hard to predict whether the server that runs LKML would see more
or less pain under my proposal.

Let's assume a straightforward/naive implementation on both ends.
That is, LKML server accepts incoming email to list (however it does
that today -- same filtering, etc.) and sends it right out to all the
members on the list.  Recipients accept incoming email from LKML
server just as they do today -- with whatever filtering they might
have in place, though, presumably, many would whitelist that server
(as they probably do today) -- except they have the additional ability
to respond indefinitely (or not at all) to the incoming email.

So far, that's within a small scalar factor of my proposal in terms of
efficiency.  (That is, it might be more efficient than today's system,
but probably not enough to justify deployment costs alone.  A system
that is inherently more resistant to UBE is the Holy Grail, especially
on this list, since it makes *everything* better.)

In cases where recipients whitelist the server, they respond to LKML's
transmission with "I've accepted responsibility", so there's no need
for LKML to track or worry further.  That's only a little better than
SMTP can do today -- again, because of simple protocol speedups.

LKML then sends a tracking request out to each of the remaining
recipients about a minute later.  Assuming they work pretty much as
they do today, they'll have committed the messages to disk, done any
preliminary O(N)-type scanning of content, and accepted the messages,
so they'll respond "I've accepted responsibility".

Only two more tracking request per remaining recipient would be sent,
assuming the recipients didn't respond.  The LKML server would,
presumably, treat inadequate response by any recipient after the final
tracking request as it would if it received a bounce in today's
system.

How does this compare, in terms of bandwidth and efficiency, to
today's sytem?  Besides recipients having an easier time of it, as
they obviously would?

Well, the LKML server must deal with incoming legitimate bounces,
which are a grab-bag of formats and sizes.  For a sufficiently large
mailing list, it *will* get such bounces.  It must distinguish between
"I'm on vacation this week" messages (which are not really bounces)
from the legitimate "mailbox disappeared" ones.

And bounces, besides being inherently oversized, often come with
payloads -- the original message -- which, in this case, LKML hardly
needs to be reminded of, so that's a waste of bandwidth on both ends
right there.

Plus, LKML must deal with joe-job bounces all the time -- undoubtedly,
spammers would forge email to come from it in order to get past naive
filters.

Since handling an incoming bounce is inherently much more expensive
than sending a tracking request and receiving a response (or waiting
for one that doesn't arrive), the question is how does the ratio of
all those tracking requests and responses compare to the (presumably
smaller) number of actual bounces?

The third-party costs are lowered as well, because sending bounces or
DSNs requires DNS lookups in the *reverse* direction.  That's
conceptually a waste of resources (DNS caches), since bounces/DSNs
aren't really *original* communications, they are *responses* to
communications.

Tracking requests and responses would not be DNS-addressed -- they'd
be more ephemeral than bounces.  (After all, sending a bounce requires
far more handshaking between point A and point B than point B sending
a single UDP packet containing a tracking request and point A
responding by sending a single UDP packet back.)

Still, if tracking ends up being much more expensive overall than
today's bounce handling in cases such as LKML, *relying* on it (by
providing no alternative in the protocol) would be a mistake.

But another possibility presents itself, as I've alluded to before.  A
new protocol could essentially eliminate *inefficient* bounces
(mainly, payload-carrying bounces), replacing them with
highly-efficient versions of SMTP's DSNs.  They'd still require DNS
lookups in the reverse direction, and perhaps even a TCP connection
(not just flinging a single UDP packet back at the original sender) to
ensure the sender received the DSN.

List-managing software could dynamically decide that, based on its
profile of recipient availability and other factors, it is best to
switch from tracking to requesting such bounces -- and back again, as
circumstances warrant.

Some recipients might choose to never return such bounces under the
new system (and in fact they might choose, or be unable, to do so
under today's SMTP, despite language in the RFCs that appears to
disallow that).

However, if the *protocol* allows it, and if LKML sends its messages
to the list such that they request these low-overhead DSNs *only* in
cases of failure, it can wait, say, two weeks for such a bounce before
assuming each recipient got the message, just as it does *today*,
using a new, and modestly more efficient, protocol.

If recipients are insufficiently willing to send such bounces and the
LKML server still wants to attempt to verify receipt, it can send a
single tracking message per unconfirmed recipient after that two-week
period has expired.

That makes the incremental cost of running my new system, over today's
SMTP, pretty much unmeasurable, even for the LKML.  (This ignores the
one-time adoption cost.)

Further, a list manager like LKML might even choose to request DSNs as
of a particular level of success *or* failure, so it would tend to be
notified earlier, by recipients whose systems cooperate, that a given
message has in fact been received, in case it wants to be more
aggressive about trying alternate routes, deliveries, etc.  (For some
list members, e.g. Linus Torvalds, that might actually make a lot of
sense.)

So, comparing apples to apples and oranges to oranges, I don't believe
my new protocol would be any *less* efficient than today's SMTP,
though it's "foundation", or "core competency", would certainly
involve tradeoffs.

Hence it seems best to extend that foundation by allowing something
akin to today's bounces/DSNs.  It doesn't seem worthwhile deploying
such a system without giving senders and recipients the option to
agree to not bother with tracking and, instead, use a DSN mechanism
similar (but superior overall) to today's bounces and DSNs.

>  -spc (And what's with the tracking anyway?)

Not sure what you mean by that.  Don't you track important packages
you send via FedEx &c. using their online web sites?

It does seem as though IM2000 wouldn't provide any tracking at all,
beyond an indication to the sender of whether the message was
unpinned, and possibly of who (as in IP address) has retrieved the
message.

Such information seems to require some kind of handshaking between the
sender and the message store to take place, however -- it seems too
"remote" to me to be sufficiently reliable.

That's why I keep coming back to wanting direct MUA<->MUA
communications, which takes me away from *presuming* there'll be large
monolithic third-party systems, stores, data bases, etc. that enable
email exchange.  (Although, in practice, even under my proposal, such
entities would likely exist, though more to add value in various ways
than to merely *enable* email exchange, as IM2000 and, increasingly,
SMTP seem to do.)

-- 
James Craig Burley
Software Craftsperson
<http://www.jcb-sc.com>