Re: IM2000 RNASP -- identifying message stores to recipients

>JCB> OTOH, SMTP can more easily tie up offending site resources by
>JCB> tarpitting incoming SMTP connections, at the SMTP protocol 
>JCB> level as well as at the TCP level for each such connection.  
>
>On the gripping hand, the reason for this sort of Judo is that with SMTP-based
>Internet mail the malicious sender's end is the client and the protocol is
>client-driven.  With, say, MSRAP in IM2000, the protocol is similarly
>client-driven but the malicious sender's end is instead the server.  So the
>idea simply doesn't apply.

Exactly.  The IM2000 parallel to SMTP using spare resources to tarpit
clients is to make gratuitous requests for message contents from known
sources of spam and vermin, and tarpit the resulting packets (again,
using spare resources).  It's different in that it's more of a thrust
at an opponent's "store" than with SMTP, but it'd have a similar
overall effect, that of actively punishing purveyors of spam and
vermin, in ways that impinge on their ability to serve up their
malware to entities that aren't yet aware that such punishment is
warranted.

>JCB> It seems superfluous to demand that an outgoing message store
>JCB> also have a domain name, which would imply that every "pull"
>JCB> operation require a DNS lookup even when it isn't actually
>JCB> needed.
>
>Solutions to this have already been invented.  Every web page fetch (with a
>domain name in the URL) over HTTP 1.0 in theory requires a DNS lookup.  This
>is ameliorated both by local caching proxy DNS servers that cache the DNS data
>and by the ability of the web browser to employ HTTP 1.1 persistent
>connections.

Um, no, neither case is a "solution", merely an optimization with
limited benefit, *especially* in cases like this.

Caches work only in situations involving sufficient locality of
reference.  So they work for WWW (A records) and outgoing email (MX
records) only to the extent that the "working set" of domain names is
relatively small over time -- which is the usual case for such
references, since they're largely user-driven.

When locality of reference goes out the window, caches become little
more than mechanisms by which latencies are increased.  (As I "proved"
to a certain well-known workstation vendor, who thought the reason a
3rd-party digital-logic-simulation compiler's code didn't speed up as
much as expected on its new workstations was because the code
generator didn't utilize prefetching and related technologies.  Turns
out simulating digital logic often has poor locality of reference;
it's ultimately driven by test vectors, which are inherently hard to
predict.  When I added prefetching to the compiler and tried it out on
some code they believed would "obviously" benefit, it yielded little
more than a 2% improvement.  So the problem with the new workstations
certainly wasn't CPU and bus speed, which was great, or even cache
size, which was reasonable; it was memory latencies.  Seymour Cray,
RIP.  ;-)

For example, I think using SPF to determine whether arbitrary incoming
email might be forged runs afoul of this problem.  Even DomainKeys
poses some risks here, if every incoming email that does *not* use it
results in a DNS lookup to determine if the (claimed) originating
domain says it *always* uses it.  Paul Graham's (sp?) "Filters That
Fight Back" proposal had similar problems: doing HTTP lookups on
"http://..." URLs in incoming emails, even if just to soak up
bandwidth on spammer sites, would pollute the local DNS cache with all
sorts of stuff unrelated to what local users really wanted to work on
-- that, and, ultimately it just recursed on the whitelisting problem
for web pages.

Poor locality of reference in DNS lookups is therefore much more
likely with IM2000 than with SMTP (as message-delivery models),
because it's not the *local* user base that is determining the set of
domain names to look up, it's the rest of the world that is
essentially submitting arbitrary domain names for *your* system to
look up via *its* DNS cache.

Having a limited-size local cache for such lookups would be okay as
long as a) it doesn't impinge on the DNS cache used for more
locally-directed lookups and b) for cache misses, it doesn't inquire
of an upstream cache, rather it goes directly to the authoritative
sources for information, so *upstream* caches aren't burdened with
arbitrary lookups likely to have poor locality of reference, again,
leaving them free to serve local users with their more-directed
searches for information.

Of course, this would tend to make such lookups take longer.  At some
point, we're back to the fundamental objection to the IM2000 model:
people won't like the delays they often experience when they go to
read the contents of a message whose summary information their MUA has
already displayed for them, and such delays might be substantially due
to slow DNS lookups.


As to persistent connections:

>Indeed, MSRAP was designed with the capability for multiple transactions per
>session right from the start.

Certainly, IM2000 could use persistent connections to implement
parallel (unrelated) same-type transactions in a single connection, so
a bunch of notifications could be grouped together, as well as a bunch
of message fetches (in a different connection), as well as a bunch of
unpinning requests (in yet a different connection).

But if you're claiming that IM2000 is going to avoid the problem I see
with each "pull" operation -- triggered at an arbitrary time by any
real-live human reading an email, and directed at a message store with
an arbitrary domain name -- by utilizing persistent connections, well,
I fail to see how that's going to work in practice, sufficiently to
avoid the DNS-lookup-delay problems.

So I wonder how you can be so sure it'll avoid the problem of poor
locality of reference without ginning up some experiments and
measuring the results (e.g. use tiny caches in DNS simulations to go
along with presumably-tiny samples sizes in your message-exchange
simulations).

And if persistent connections will help IM2000, then the fact that
SMTP almost always involves (or should involve) a persistent
connection to implement most or all of the notification, content
transmission, and status report (at least as far as "unpinning" the
message) activities for any *single* message, avoiding any return DNS
lookup to transmit the content, suggests that SMTP already has the
upper hand in this regard, without the need for new caching-proxy
technology.

(Of course, SMTP already has the concept of multiple message exchange
within a connection, in that a single connection can be used to
transmit several unrelated messages.  Just because qmail doesn't use
it when sending messages doesn't mean it isn't useful, since,
presumably, other popular MTAs do make use of it; which makes me even
more curious as to how IM2000 is going to manage to use a single
connection, or even a single DNS lookup, to group together multiple
unrelated MSRAP transactions?)

One thing I'm fairly sure of: to the extent IM2000 *does* fulfill the
promise of making UBM a thing of the past, this will be *less* of a
problem, because UBM is a substantial source of message content coming
from arbitrary hosts.  So some of the poor locality of reference for
incoming SMTP messages is due to UBM.  (The pertinent questions being,
about how much is due to UBM, and, will IM2000 really knock this down
*enough* to remain usable as it hits sufficiently widespread
deployment?)

>A more compelling argument for preserving the ability to include raw IP
>addresses and port numbers in notifications is the "I've got a small
>organisation and I don't want to be bothered with DNS service when the machine
>on my network running my own private message store has a well known fixed IP
>address that I could just plug in directly and use." argument.  Of course, in
>such a situation one probably deals with private DNS service for other reasons
>(such as for a Microsoft Windows NT 2003 network) already anyway.  So the
>argument can often be a weak one.

Not weak enough, IMO, to rule out a reasonable use for it.  But that's
pretty much orthagonal to the performance issue, to the extent people
(especially spammers) will publish domain names anyway.

What publishing IP addresses allows is for sites with well-known,
mother-approved outgoing message stores to help reduce the load on
everyone else's DNS by forgoing its use to look up its messages.
Kinda like giving out your phone number instead of just your name and
address (which requires at least one phone-book lookup), it might be a
nice thing to do.

But I don't think a decision one way or the other will be what enables
or destroys the ability for IM2000 to succeed in the long run.

-- 
James Craig Burley
Software Craftsperson
<http://www.jcb-sc.com>