Re: ROI on IDS/IPS products

I'm going to try this one more time and then just let it go. Just  
because the current crop of products suck doesn't mean we need smarter  
users.


On Mar 2, 2009, at 1:36 PM, Jack Whitsitt wrote:

> I don't normally chime in here, but this seemed to warrant a response:
>
>
> So, why do you consider it so far fetched that I might configure an  
> IPS not on a signature-by-signature basis but an application,  
> resource, and risk basis?
>
> Because risk - or more specifically, risk appetite - is a business  
> measurement of business functions.  The risk associated with  
> something getting through and compromising that Windows server,  
> switch, or group of workstations is not something your vendors will  
> ever, ever be able to tell you.  Even if the technical capabilities  
> of IPS (or any security device) were perfect (see below), you'd  
> still hit this wall.

I absolutely agree with you. Let me put it this way. The best products  
will solicit the input of local knowledge the user is likely to have  
and combine that with the expertise embedded in the software and the  
content to deliver a solution that would not be possible from either  
alone.

So, let's say I'm running a web farm. I can park an IPS in front of  
the web farm to protect it. I know that HTTP access to these servers  
is my bread and butter and I cannot risk an interruption in service. I  
spend a lot of time and money making sure these applications are  
patched and not vulnerable to basic attacks. I would call this a high- 
availability service. I also use SSH to remotely log into and manage  
my web boxes. If I cannot remotely log into my web boxes from the  
Internet I can always drive into the office to fix things but I don't  
like to do that. In addition SSH access has the potential to give full  
access to these servers.
If my pretend IPS existed then I'd configure it to only block HTTP  
traffic if the vendor has rated the signature as being 95% reliable or  
better. Signatures with a reliability of 80-95% should send me an  
alert and everything else should just be let through as I can't  
tolerate the log load.
SSH, on the other hand, should be configured to block on any signature  
with a 50% confidence or above and log everything else. I don't SSH  
into the boxes all that often so the log load should be minimal.
When the vendor releases new content it is applied using the same  
thresholds as above.

This is a slightly simplistic picture, of course, because I may want  
different thresholds depending on what the signature is supposed to be  
blocking. I'm more paranoid about things that can result in full  
breach versus those that are just DoS attacks, for example. Hopefully  
you'll give me enough leeway to send a simple e-mail rather than  
provide a full product requirements document. I need to leave  
something for the IPS Product Managers to do after all.

In summary, I provide the local risk tolerance and application usage  
because the product cannot know my business or my risk tolerance. The  
vendor, through the product and its content, provides the assertion of  
reliability because I cannot know how effective a signature the vendor  
was able to provide.

Now, some are going to read this and say "But product X already allows  
me to do this configuration." Yes, some do but they are generally  
lacking in two areas, the first is the ability to specify the policy  
based on applications and resources and the second is the required  
trust in the vendor and the vendor's openness in communicating the  
efficacy of specific detections.
>
>
> However, as I've said, this, trust already exists.
>
> That trust exists only after local testing. If you turn on IPS  
> signatures without testing them locally, well, then....good luck to  
> you....because:  Things aren't, unfortunately,  nearly as  
> technically cut and dry as you seem to believe they are. In some  
> cases, there is traffic that you can be...95%? ...sure is bad in all  
> cases...but that's a very small subset of what goes on over the  
> network. Good traffic is not so easily and predictably  
> distinguishable from bad most of the time and some level of local  
> business knowledge and technical testing is required for good  
> deployments.

I'm not suggesting that you would blindly buy a product and deploy it  
without testing how well it does in your environment. As a matter of  
fact if the product I outlined existed it would need to be in  
evaluation through a least a couple of content updates to make sure it  
was behaving as expected. This is different than needing to tweak an  
IPS on the first Tuesday of every month.

I'm also not naive enough to think that a product would be truly  
maintenance free. Effectiveness reports will need to be run. Some logs  
will need to be analyzed to make sure that things are not slipping  
through, and, if good traffic is blocked, remediation may need to  
happen.
>
>
> I'm not sure how this isn't obvious? How many times have you seen  
> activity that's legit on one network be a sign of something  
> dangerously wrong on another?

Truly legit traffic on one network being a sign of something wrong on  
another? I can make up any number of scenarios to make that true but  
they would all be stretching reality.

Really? Are you suggesting that completely legit traffic on one  
Internet facing web farm would be considered malicious on a different  
Internet facing web farm? Or are you suggesting that BitTorrent may be  
allowed on one network and a sign of a problem on another network? I  
see the first as highly unlikely and the second a matter of  
application policy.

So, I agree that customer expectations and product capabilities are  
currently very far apart. However, I think it is the products that  
need to come forward instead of expecting every IPS customer to hire  
their own team of network security analysts.

-J