Re: PR Problem?

Todd A. Jacobs wrote:
> Well, ideally this is where hybrid systems with DNS records and such
> would come into play, but in a more realistic and practical system I
> think one just defines a best-practice. That's why I tossed the idea of
> basing the "standard policy" on the number of addresses in the mail out
> there: because the MUA *does* know how many addresses are involved.
> Granted, that doesn't mean the destined recipients will check, but if
> they did they should probably be following the "recommended standards,"
> whatever they end up being.

It would be interesting to see if one could define best practices and 
then make it possible to some mechanism to let the sender match the 
receiver's needs.  If the standard practice becomes score fudging, I'll 
probably just force the stamp to be large enough so that it overwhelms 
whatever scoring mechanism underlies the system.

It will also be interesting to see if we can model at to see if there is 
a golden window for spammers as well with the score fudging technique. 
I've closed the window elsewhere.


> You keep assuming "hybrid environment," whatever that means, but I don't
> think that's currently a practical real-world scenario. Real-world, you
> aren't slowing down the botnets, you're busting the performance of an
> ISP or e-commerce server.

Some how I get the feeling we're talking at cross angles to each other. 
  I apologize for my part of any confusion.

A hybrid system (in my definition) is a system which uses external data 
or predicate based on that data to reduce the spam generation load.

A naïve system's generates stamps at maximum rate (i.e. the same rate as 
messages coming through).

so, it's exceptionally practical in the real world.  It reduces load so 
that an ISP or in e-commerce server can generate stamps when necessary. 
  But it only works in a system where the receiving filter has a 
matching hybrid nature.  That it can accommodate only a few stamps and 
then by observing the behavior of the end user, figure out who no longer 
need stamps and treat them appropriately.

The hybrid predicates that I can see having value are: "who do I know?" 
and, "what does DNS say?"
> 
> Statistics claim that 80% of all email traffic is spam. Well, if you're
> an ISP, you want to track botnets by spam volume (e.g. reading your
> logs), not by seeing which server has just crashed because its CPU is
> pegging out at 100%.

You mean you would put a stamp generator in place that doesn't do rate 
limiting?  If the system has maxed out and the queue length has grown 
beyond 10 minutes worth of messages, I would just send the messages on 
without stamps.  No sense making enemies of your friends.

in a hybrid system, this is the T0 problem.  and the only solution I see 
is to dump stamp generation when the load gets too high.  Eventually, 
the required workload will settle down to something much more practical.

> Slow-minting is a selling point to *recipients*, not senders. So, if you
> want centralized minting, you have to come up with a better slogan than
> "we'll crash your servers if your users are sending spam." :)

:-) indeed.  What we should probably try to emphasize is the minting 
differential between spammer and defender.  Again, the hybrid broken 
record says that you minimize stamp generation while keeping the spammer 
load at the same level.  This lets you specify a larger value stamp for 
the same amount of resources used with a naïve system which increases 
load on the spammer but not so much on the defender.

> 
>> It looks like your description is of a received side chain.  Hash cash
>> would be injected prior to the first postfix.  If a stamp is valid and
> 
> Yes. That would allow a policy daemon to verify stamps outbound, too,
> which *would* be a win from an ISP or enterprise point of view.

Yes.  One could change the outbound filter chain to verify stamps 
outbound.  it would detect zombies trying to serial or parallel double 
spend.

>> present, the first procmail should detect that and abort processing in
>> the rest of the chain if successful.  Otherwise you process is normal.
> 
> See, that's sort of my point. I'm not sure that hashcash should assume
> it is the final arbiter of what is or isn't spam. I realize that at
> present all it does is mint or verify hashcash stamps; I just think it
> should be hashcash's job to append or validate the RFC 822 headers in
> filtering mode.

I disagree that's the job of hash cash.  It's only task should be to 
validate or create stamps/tokens.  If you want another library to handle 
that then, by all means, feel free to create one.

> 
> If you want to talk Unix philosophy (one tool, one job) that's a
> different conversation--although most Unix tools are capable of being
> used directly in a pipeline wile hashcash can't . But if the question is
> "Why don't more people implement this as part of their toolchain," I
> think that the inability to simply drop it in as a filter (in the sense
> of filter-processing, not mailbox filtering) is part of the reason.

don't be hampered with a data handling model that is over 30 years old 
and only works with one particular user model.

If you look at most UNIX commandline tools, they are no longer one tool 
one job.  And while one tool one job is a strength in the small, it 
breaks down in the large.  when you start trying to chain things in 
e-mail, you really need to look at it from a very different perspective. 
  I think the postfix filter is one of the best filter interfaces 
around.  It's based on an enhanced SMTP environment and have reasonable 
expectations of queuing, retrying etc..  it also lets you deal with data 
bidirectionally if you're clever.  Pipes don't do that and that's why 
they are bad for e-mail.

if you look in the twopenny blue filter chain, I have a series of 
objects all derived from the same basic filter archetype.  Then on 
customization, I create a list of objects that I can just call in a 
sequence.  As I've mentioned earlier, I would love to drag those objects 
outside of the twopenny blue post filter and let them self assemble just 
by their presence.  Unfortunately, I don't know enough.

That's the level of configurability that should be in any filter.  Not 
because he can string it together with a bunch of UNIX pipes.  But have 
real control and data handling.

>> If you need the verification header into abort processing or otherwise
>> modify it later on, then that's your own filter responsibilities, not
>> that of anybody else.  I think this falls in the category of filter
>> philosophy.
> 
> Yes and no. Most tools, including bogofilter, spamassassin, and even
> procmail support the idea of acting as a pass-through filter and provide
> error codes and header modification (although in procmail's case you
> have to shell out to formail for the latter). I'm *not* saying that
> hashcash should decide what to do with an email; I'm saying that it
> *should* be able to mark an email as containing a valid or invalid
> stamp.

if you wish to limit yourself to that model you can do that by putting a 
wrapper around hash cash as it stands today.  but that very model is 
also part of the reason why filters are the way they are today.  Over 
the past few years starting with the first challenge response plus hash 
cache system and moving onward through the pipe type oriented model you 
describe, pipe-based filters have always come up short.  Performance is 
bad and I don't have all the information I need to do more advanced 
filtering and signal analysis.  it just didn't behave the way I wanted 
it to behave.

E-mail is inherently loopy with lots of feedback possibilities.  By 
taking advantage of feedback loops, paying attention to and and using 
information on both inbound and outbound sides of your MTA/MUA 
environments, you will increase effectiveness, reduce user workload, and 
make it possible to throw the maximum load back on the spammers.

Another thing to consider is if it was useful and easy to use pipe-based 
filters, why hasn't anyone done it with hash cash, and why are most of 
the content filters operating as a internal component of the MTA and not 
as a delivery agent component?

---eric