AppSec Blog

Pentesting: Do you need "coverage" ?

Last week, I had a discussion via instant messenger. The discussion essentially evolved around the need for coverage in a pentest. It has always been my conviction that a pentest should find as many problems as possible. In my opinion, the pentest is not over once I got root on the system. In many ways, this is where it starts.

I think it is a matter of pentesting philosophy. I see myself primarily as a coder. A pentest is, for me, part of my software testing regimen. Like my other tests, code coverage is of utmost importance. I don't consider my application working until it has been fully and thoroughly tested. A pentest is just another, very special, test I subject my code to. As a result, I am not a big fan of "black box" testing, or "early dawn raids" as I call them. They may find a problem, but the tester will typically waste a lot of time finding and exploiting vulnerabilities, which may have been trivial to exploit with a bit of source code, or after talking to the developer. I rather have the pentester spend their precious time on going after yet another possible vulnerability.

I do understand that a pentest does not find all bugs, but the aim should be to do just that. I strongly believe that a pentester should be held to the same standards as a developer. Would you accept software that works once? Code that only has one bug fixed? Just because it is hard, doesn't mean that we shouldn't attempt it. In many ways, this is what I enjoy about being a developer: Hard problems.

Is it possible to come close to the goal? I think it is. All of this is based on a good framework and to apply this framework thoroughly. You do your recognizance, and don't stop at the first web application you find, unless the scope of the project limits you. Next, you map out the web application, and try to find not just all URLs or "pages," but all features. A feature you miss in the mapping phase will be a feature you miss in your test. Next, you discover vulnerabilities. And finally, you try to exploit them. Once you manage to exploit a vulnerability, you are likely to find more content and your process starts all over again.

Sounds boring and tedious? Yes it is! If you don't know how to script. I say this a lot: If you don't script, you will soon be replaced by a script. If you want to distinguish yourself as a pentester, you will have to understand your tools well enough to extend them and build upon them.


Posted October 1, 2009 at 1:08 AM | Permalink | Reply


Honestly, its very hard to level the standard between pen-testers and software QA/developers. I believe Software QA/developers focus on functionality and requirements specs as the scope of their work while pen-testers go further in a very technical level. But I agree that somehow the level should be reach or meet in some aspects and the framework must be solid to reach the level of maturity.

Posted October 1, 2009 at 3:07 PM | Permalink | Reply


WoW. Excellent article.

Posted October 2, 2009 at 1:55 AM | Permalink | Reply


you've done an excellent job describing a few issues.
First is that developers are not pentesters and usually shouldn't be. mostly because they look as the problem as an problem with a single application and not how all those issues fit together as its related to your network security posture and for protection of your critical assets/data/or IP.
Second, you've described perfectly what should go on during a vulnerability assessment and application (source) review and not a pentest.
we need to get away from "pentests" that attempt to find every possible vulnerability, those are vulnerability assessments or if you want to justify your existence vulnerability assessments with exploitation or vulnerability assessment with source code audit and not pentests.
Its time for a pentest when you've done all the things you mentioned in your posts and you think your app is good to go & no one can break it or into it. Or more importantly you think you can catch the attacker once he's in your network and one 0day doesnt ruin your whole day.
Your find every vulnerability pentest can be / should be/ and rapidly is being replaced by a monkey with a scanner or an appliance in a rack.
see chris nickerson for the rest of the rant''

Post a Comment - Cancel Reply


* Indicates a required field.