Re: How secure is software X?

On 5/12/06, David Litchfield <[email protected]> wrote:
> How secure is software X?
>
> At least as secure as Vulnerability Assessment Assurance Level P; or Q or=
 R.
> Well, that's what I think we should be able to say. What we need is an op=
en
> standard, that has been agreed upon by recognized experts, against which =
the
> absence of software security vulnerability can be measured - something wh=
ich
> improves upon the failings of the Common Criteria. Let's choose web serve=
r
> software as an example. When looking for flaws in a new piece of web serv=
er
> software there are a bunch of well known checks that one would throw at i=
t
> first. Try directory traversal attacks and the several variations. Try
> overflowing the request method, the URI, the query string, the host heade=
r
> field and so on. Try cross site scripting attacks in server error pages a=
nd
> file not found messages. As I said, there's a bunch of checks and I've
> mentioned but a few. If these were all written down and labelled with as =
a
> "standard" then one could say that web server software X is at least as
> secure as the standard - providing of course the server stands up.
>
> For products that are based upon RFCs it would be trivial to write a simp=
le
> criteria that tests every aspect of the software as per the RFCs. This wo=
uld
> be called Vulnerability Assessment Assurance Level: Protocol. If a bit of
> software was accredited at VAAL:Protocol  then it would given a level of
> assurance that it at least stood up to those attacks.
>
> Not all products are RFC compliant however. Sticking with web servers, on=
e
> bit of software might have a bespoke request method of "FOOBAR". This ope=
ns
> up a whole new attack surface that's not covered by the VAAL:Protocol
> standard. There are two aspects to this. Anyone with a firewall capable o=
f
> blocking non-RFC compliant requests could configure it to do so - thus
> closing off the attack surface - from the outside at least. As far as the
> standards go however - you'd have to introduce criteria to cover that
> specific functionality. And what about different application environments
> running on top of the web server? And what about more complex products su=
ch
> as database servers? I suppose at a minimum for DB software you could at
> least have a standard that simply checks if the server falls to a long
> username or password buffer overflow attempt and then fuzz SQL-92 languag=
e
> elements. It certainly makes standardization much more difficult but I th=
ink
> by no means impossible.
>
> Clearly, what is _easy_ is writing and agreeing upon a VAAL:Protocol
> standard for many different types of servers. You could then be assured t=
hat
> any server that passes is at least as secure as VAAL:Protocol and for tho=
se
> looking for more "comfort" then they can at least block non-RFC compliant
> traffic.
>
> Having had a chat with Steve Christey about this earlier today I know the=
re
> are other people thinking along the same lines and I bet there are more
> projects out there being worked on that are attempting to achieve the sam=
e
> thing. If anyone is currently working on this stuff or would like to get
> involved in thrashing out some ideas then please mail me - I'd love to he=
ar
> from you.

why do we need this?

you're referring to what already takes place commercially.

"hi i want a security assessment".

who's going to do these assessments for free? who confirms that the
people doing the assessment know what they are doing?

"Customer: I was hacked .." -> me: -> "David Litchfield told me it was
secure, blame him" -> "David Litchfield: Oh no, our VAAL is just a
guide." -> "Customer: So why the hell do I care about it then?"

Guides for people to use are okay (hello OWASP Guide, and others) but
all your trying to start is a non-commercial free security assessment
service.

... ?

-- Michael

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/