Re: PCI DSS & Firewalls

Isn't the point of pen-testing to take up an attackers' perspective and 
hit all your defenses to see if you missed something or misconfigured 
something?  I mean, unless you're the only person who set up 100% of 
your infrastructure, how are you to know that someone didn't 
accidentally leave telnet open?  If you didn't write 100% of the webapps 
your company is using, how are you to know they don't have SQL injection 
flaws?

Marcus J. Ranum wrote:
> Frank Knobbe wrote:
>  >> I also agree with Marcus that it's the Pen Tester's Employment 
> Security
>>> Act..
>>
>> Wouldn't you want to test your security controls periodically?
>
> Of course. That's part of good engineering. But...
>
> Good engineering says that you have structural elements that
> should have various known and measurable capabilities. In
> security, that would mean that you have a security design,
> and that design would call out specific properties of how
> the system should work and should behave. Yes, you'd want
> to test to verify that the system was still working in
> accordance to its design.
>
> That's exactly the opposite from periodically flinging
> poop at it and seeing if it still smells like a rose
> afterward.  Pardon my metaphor. :)  The idea of pen testing
> IS TO SIMULATE AN ATTACK
> well, your design ought to be such that no known attacks
> will work against it. Put differently
> THERE SHOULD BE NO KNOWN POINT OF ATTACK
> If that's the case, then simulating an attack, using
> all the known tricks in the bad guy's arsenal - is
> utterly stupid. If what you were to do was to perform a
> top to bottom verification that the system's implementation
> was still in accordance with its specifications
> then that's a "design review" coupled with an "implementation
> test" or "design oriented implementation review" - doing
> that sort of test would require a completely different
> set of tools from what a pen tester uses, and it would
> be performed with a system design document in hand, from
> the "inside" toward the "outside."
>
> Of course the bad guys are innovating too, and it's very
> much worth keeping track of what they're up to and updating
> designs and plans accordingly. But - again - that doesn't
> need pen testing; that needs periodic design reviews in
> the face of newly uncovered forms of attacks. I.e.: your
> system should be proof against SQL injection attacks; and
> your code should have been carefully reviewed and tested
> to be in accordance with that design. If you want to do a
> "pen test" at that point, they should be looking at your
> source code, not badpacketing you or whatever silliness.
> If the bad guys invent a new form of attack, then it's
> time to review your design to see how it resists that
> form of attack: defend against general CATEGORIES
> not SPECIFIC INSTANCES.
>
> The pen testing paradigm is intellectually bankrupt.
>
> mjr.