When purchasing (or performing) a security assessment, knowing exactly what you want (and what you provide) is very important. With a myriad of various engagements, it can be challenging in deciding on what is best for your organization.

That’s why I decide to write this diary as a small guide on how to decide which security assessment you want, and (what’s even more important) what you should expect to receive.

From technical point of view, I generally categorize security assessments into the following three categories: vulnerability scanning (assessments), penetration tests and red team exercises. Deciding what you want to perform/purchase should go in exactly that order, and depend on your organization’s security maturity level. Let’s see what each of these is about.

Vulnerability scanning (assessments) is something that we should be doing regularly. The goal of vulnerability scanning is to find low hanging fruit – vulnerability scanners will do a great job in enumerating installed patches, finding default accounts and misconfigurations. This is the first step in vulnerability management, and no matter how big (or small) your organization is, you should be doing this regularly.

While majority of organizations will perform vulnerability scanning on their own (with their own tools), sometimes I see clients that ask me to do this on their behalf and provide them with a shortened executive summary – one problem that vulnerability scanners might have is with false positives.

Penetration testing is the next step. When deciding on a penetration test, scoping is very important: the goal of a penetration test is to find all (well, as many as possible) vulnerabilities in the target scope. The target scope can be a set of IP addresses or networks (when we talk about a network penetration test), a web application, web services and so on.

Since a penetration tester is normally limited in time (typically penetration tests last between 5-15 work days), it’s obvious that the whole process should be as efficient as possible. This means that the scope should be define as good as possible, and that obstacles should be removed. For example, if you are testing a mobile application, it would be beneficial to provide a build without obfuscation for your penetration tester: we know what obfuscation can be circumvented and by providing such a build, the penetration tester will not waste time on deobfuscation, but on finding vulnerabilities.

Additionally, a penetration test will also test for vulnerabilities that a vulnerability scanner cannot find – one example of such vulnerabilities are logic vulnerabilities. As we still do not have AI capable vulnerability scanners (hey, I mentioned AI, now I just need to put in this diary machine learning and blockchain and we have everything covered :), logic vulnerabilities are always missed by automated scanner. Let me give you one easy example: you are using your Internet banking mobile application to perform a transaction of -100 EUR. Vulnerability scanners will normally miss such vulnerabilities since they lack understanding of the context.

The result of a penetration test is a report that should detail all identified vulnerabilities, with recommendations on how to fix them. All listed vulnerabilities should be verified by the penetration tester – there should be no false positives there!

As you can see, penetration tests require a lot of manual work – that’s why they last long(er) and are typically more expensive.

Finally, a red team exercise is the ultimate test of your defenses. In a red team exercise, the attacker(s) are given a final goal and they can typically use anything they want to achieve they goal. They might be writing new exploits, using social engineering, moving laterally …

You will see the main difference between a red team exercise and a penetration test: with a red team exercise there is one ultimate goal, while a penetration test aims to find all vulnerabilities in the defined scope. A red team exercise might miss some vulnerabilities and never report them, but it will show you how you stand against a real attacker. Quite often at the same time blue teams are tested – this will show how good you are in detecting attacks and potentially preventing them while they are happening.

To wrap this up – depending on what you have done to manage vulnerabilities in your organization and ramp up defenses, pick which offensive engagement suits you the most. If you have never performed a vulnerability scan, or you do not regularly test for vulnerabilities, it makes not sense to run a red team exercise – you are wasting money if a red team (or a penetration tester) comes in and obtains a domain administrator’s account in 30 minutes. So go step by step, and slowly improve your defenses.

Do you have your own war stories or want to share comments? Let us know!

--
Bojan
@bojanz

INFIGO IS