Re: Up and running on the real sever - and I

Dennis Wynne wrote:
> I should have explained my doublet-triplet idea better. What I was 
> thinking is - once a triplet has passed the timeout check and had mail 
> allowed through on that triplet, THEN allow that sender to send to any 
> of my users from the same IP with the same From: ID. If I just save 
> the from and IP then I could get fooled easily by a SPAMmer, but if I 
> make sure that a valid triplet exists THEN allow the doublet to work. 
> I may not be a great idea anyway, since it would involve an extra 
> database query or two. You would check for an exact match on this 
> triplet and if that fails then check for a
> PASSED match on the doublet (from and IP) and it that fails you would 
> temp fail it and add it as a triplet to the database.

Sounds like what I do. You didn't mention what happens to the database 
when there is no triple match, but there is a double match.

In this case you might want to add a new triple to the database for the 
message in hand. In my system I add a special type of record (MIDGREY), 
which will immediately allow any future messages matching that full 
triple, without reference to the record where we originally saw the 
double match. However because we haven't seen two instances of the 
triple, this MIDGREY record cannot, in and of itself, be used as a 
target for a double match.

If then a message comes in matching the full triple of a MIDGREY record, 
it is accepted, and the record is updated to a LIGHTGREY record.

Ah. That will only make sense if you know what a LIGHTGREY record is. 
Right. When a completely fresh triple comes in, we defer the delivery 
attempt, and create a DARKGREY record. If a triple comes in which 
matches a DARKGREY record, and it's after the greylisting time has 
elapsed since we first saw the triple, we accept the delivery and update 
the record to a LIGHTGREY record.

So we allow double-matches on LIGHTGREY records because those records 
are ones where we have definitely seen two attempts.

(For completess, the other record types are BLACK, WHITE, and REVERSE. 
REVERSE is for recording outgoing deliveries so that we can immediately 
allow incoming deliveries where we have seen a matching outgoing message.)

> I still we be looking at some queries, since I will be expected to 
> produce a list of orphaned triplets there never passed and never were 
> retried - just to make them happy :-)

This may not be helpful because the list you allude to here will 
approximate a list of spam attempts; a haystack in which any false 
positives will be needles.

> But turning it way down on the retry time should answer a lot of user 
> concerns.
>
> BTW, I was trying to Google up the SMTP RFC to see what the "official" 
> SMTP rules are for retry time, delays, etc - but could not find what I 
> was looking for. Is this even listed in the specs or maybe suggested?

Remember that although the RFCs are good guideline, any e-mail 
administrator working for some organisation will ultimately be 
responsible for making the best result for that organisation. The RFCs 
are useful as a starting point, and for quoting at people *under certain 
circumstances* (e.g. when you are trying to explain to someone why they 
mustn't put a dot at the end of an e-mail address, or dealing with 
remote e-mail administrators), but slavishly following them is not 
always the best path.

For example, I reject delivery attempts where the HELO parameter is not 
valid in certain ways. What I found was that many otherwise valid 
delivery attempts used a HELO parameter which was just a single word. 
Repeatedly having to deal with these "false" positives was not giving 
best value to my employer's shareholders, so I relaxed that specific 
rule. (I still reject, for example, HELO parameters which are dotted 
quads without the required surrounding square brackets.)

So the RFCs may say that retries should not be done more often than X 
minutes (say 15), so you might say "ah well then I can just set my 
greylist time to 14 minutes, because anything less is invalid according 
to the spec - ha ha ha". That's all fine and good until you get an 
important message from your most important customer deferred for 4 
hours, just before an important meeting, because of a pathological 
interaction between your companies' mail servers. Your CEO will not be 
too impressed when you start quoting RFCs at him!

I haven't been doing analyses (and I realise that this is totally 
unscientific), but thus far I haven't felt that we are accepting a 
significant extra amount of spam because our greylist time is only one 
minute.

> I just had my first SPAM to me get through the system. It is 
> disappointing, but still better that before the new server was in place.

I use greylisting as part of a stack of several layers of e-mail 
processing. I think last time I looked I had about ten different layers 
--- not quite as impressive as it sounds, some of them are just a single 
line in the MTA configuration.

Bill
-- 
The contents of this e-mail and any attachments are confidential and may 
be legally privileged.  If you have received this e-mail and you are not 
a named addressee, please inform us as soon as possible on 
+44 118 901 2999 and then delete the e-mail from your system.  If you 
are not a named addressee you must not copy, use, disclose, distribute, 
print or rely on this e-mail.  Any views expressed in this e-mail or any 
attachments may not necessarily reflect those of Tao's management.  
Although we routinely screen for viruses, addressees should scan this 
e-mail and any attachments for viruses.  Tao makes no representation or 
warranty as to the absence of viruses in this e-mail or any attachments.  
Please note that for the protection of our business, we may monitor and 
read e-mails sent to and from our server(s).