Re: Future of Qmail-LDAP

50% reduction of bandwidth can be explained only by more aggressive RBL,
firewall rules based on RBL data or traffic offloading. There is no other
way. All other filters are based on content scanning or increase amount of
traffic. Cisco Ironport advert video shows that they split traffic into
blacklist, greylist and whitelist queues. blacklist reduces amount of
traffic. whitelist reduces processing. senderbase (most likely something
based on spamtraps) catches outbreaks and adjusts
blacklist/greylist/whitelist queues. You can't protect your customers from
spam and virri like that ad says. You just reduce amount of spam to
tolerable levels and hope that you don't get too much complains from false
positives.

Now you pay for service without actually knowing how it works and you will
be forced to pay same amount or more after your service contract expires.

I use simscan instead of qscand. you don't have to implement quarantining,
if your spam scanner follows email protocol and bounces emails back to
sender. RBL based filters don't offer quarantine. If I remember correctly,
those Ironport boxes run OpenBSD and Postfix or some other smtp proxy. If
they use firewall to filter spam and get that reduction of traffic by
cutting out large subnets, they are dealing with it in most unfriendly
way. These blocked networks don't inform sender about delivery failure
immediately. They do that only several days later.

It is possible that I don't estimate scale of your setup. I usually work
with SOHO setups. I just don't like to pay millions for something without
understanding how it works and that is not rocket science. I don't like
when people say that some cool and expensive commercial product gives 50%
boost of something. You can't get that kind of boost without making
serious safety cuts or you boost something that is not optimized.

2010.02.09 09:49 Scott Ryan rašė:
> You say I could have done myself. True - it's not rocket science. But
> the time saved, the costs, the effort involved in both building and
> maintaining what would have been a monster of a system. The
> requirements from business also included quarantining mails - a tricky
> proposal with 1.5M qmail-ldap system.
>
> Up until we put in the ironports, we had built our own scanning
> application used by qscanq which could pull in spamassassin, clamav
> and any other scanning/filtering service - but it is a lot of work and
> resources. And to give an idea - it is not possible to run
> spamassassin on such a huge scale without serious additional
> resources.
>
> We also used RBLs inc spamcop. But they are only effective up to a
> certain level. It is also a bit naive to think that Ironport is
> basically just spamcop.
>
> Greylisting? Seriously for that amount of traffic? I had 12 MTAs - all
> of which were at least quad CPU machines (HP DL580s etc) To implement
> a highly available database with sufficient storage both in size and
> speed would be of significant costs - never mind the additional
> overhead of managing it.
>
> Tomas, it is easy for you to say you could have done it yourself but
> if you could present hardware, management and the software that could
> do the above for less then you could be well on your way to making a
> lot of business for yourself.
>
> Over the three year life cycle (minimum) it turns out to cost less
> than $0.03 per user per month.
>
> Bargain.
>
> On Mon, Feb 8, 2010 at 10:57 PM, Tomas Kuliavas
> <[email protected]> wrote:
>> There is nothing magic in spam filtering. They either use some custom
>> RBLs
>> or combine standard RBLs with content filtering based on some open
>> source
>> software (most likely spamassassin). Even conservative RBLs classify
>> 75-90% of email traffic as spam. Since ironport owns spamcop, then their
>> RBL data can be based on spamcop reports. Spamcop is not conservative
>> RBL.
>> I think they even had page which stated that their RBL should be used
>> only
>> for tagging.
>>
>> You paid 1.5M for dedicated spam filtering box which you could setup
>> yourself.
>>
>> If your traffic is reduced, then their solution is based on RBL or some
>> RBL data is integrated into firewall or you didn't use RBLs in the first
>> place and relied only on greylisting/content filters. Or they offload
>> email traffic to remote servers and only your own traffic is reduced.
>>
>> 2010.02.08 22:47 Scott Ryan rašė:
>>> To handle over a billion mails a month and reduce bandwidth by about
>>> 50-60% (in a country who's international bandwidth is limited compared
>>> to rest of world) as well as severely reducing the load on the SAN and
>>> mail storage environment - all in all it was about $1.5M.
>>>
>>> Turned out to be a no brainer.
>>>
>>> On Mon, Feb 8, 2010 at 8:49 PM, Hugo Monteiro
>>> <[email protected]>
>>> wrote:
>>>> On 02/08/2010 07:39 PM, Scott Ryan wrote:
>>>>>
>>>>> I ended up handing the fight over to Ironport. Put in some Ironport
>>>>> devices in front of my mail environment and ...
>>>>> VIOLA! no more spam.
>>>>>
>>>>>
>>>>
>>>>
>>>> Reputation filters .. isn't that a new fancy name for RBLs? Thanks but
>>>> no
>>>> thanks.
>>>>
>>>> Out of curiosity.. how much $/€/£ each, and how many did you need for
>>>> your
>>>> infra-structure?
>>>>
>>>> R's,
>>>>
>>>> Hugo Monteiro.
>>>>
>>>> --
>>>> fct.unl.pt:~# cat .signature
>>>>
>>>> Hugo Monteiro
>>>> Email    : [email protected]
>>>> Telefone : +351 212948300 Ext.15307
>>>> Web      : http://hmonteiro.net
>>>>
>>>> Divisão de Informática
>>>> Faculdade de Ciências e Tecnologia da
>>>>                   Universidade Nova de Lisboa
>>>> Quinta da Torre   2829-516 Caparica   Portugal
>>>> Telefone: +351 212948596   Fax: +351 212948548
>>>> www.ci.fct.unl.pt             [email protected]
>>>>
>>>> fct.unl.pt:~# _
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Scott Ryan
>>> http://bonoboslr.wordpress.com/
>>>
>>
>>
>>
>
>
>
> --
> Scott Ryan
> http://bonoboslr.wordpress.com/
>