First a Few Words About the Use of the Term "APT"

Advanced Persistent Threat (APT) has been getting a lot of press in the past year. There is plenty of debate over whether APT is a "who" or is a "how." However, there is little debate over APT's buzzword status. This is how I'm using the term today: as a buzzword to get you managers' attention and to get them to the table for this exercise. Like all security efforts, if you lack management support, you are likely to fail.

Why you Need an Exercise

An APT occurrence is a low-frequency high-impact incident. You are not likely to have policies, procedures and run-books prepared. There are not a lot of people who have first-hand experience dealing with it, and they will be hard to find when you actually need them. This is why you need a flexible response framework and run through scenarios like these to stay prepared.

Before the Exercise

You will want to prepare the appropriate communications to get your message out to management and the participants, this can be as simple as emails, but my require full reports or presentations depending on your organization. The message should minimally include a description of the threat, and how it may affect your firm. Or, "what is APT" and "why should we worry?"

Describing APT-like events

You can Google around and find some really great bloggers discussing APT. Management won't know these people so it might not be the only source that you want to include. I'd recommend media reports (especially the media that your management reads) about the following high-profile events:

• Google Aurora
• NASDAQ
• Night Dragon
• RSA

Why Should We Worry?

This is the classical "why should I care" question which you're used to dealing with. If you're not on the obvious-targets list already you may want to point out that every organization has its secrets, and other groups probably want those secrets, either for direct profit or to simply post them on the Internet to make you look bad. These groups may employ tactics similar those high-profile events, and that's the purpose of this exercise, to determine if you're prepared to handle such attention.

Preparing the Notification

You will start off the exercise with the "notification." Undoubtedly your organization will become aware of the event through a visit or phone-call from a government or law-enforcement agency. They will contact your upper management, not your security team or help-desk. This will not be a good day for any of you, so again, it's a good idea to involve upper management in this process so they know who to contact after they are made aware of the incident.

The scenario will be something like: "we have learned, by way of an informant, that your crown jewels have been stolen, we think the compromise occurred over six months ago and the attackers possibly used one or more of the following IP addresses," and they provide a list of 62 IP addresses.

Always include a list of things to look for and the list will contain many false-leads, and probably be difficult to leverage in your environment. It's not uncommon to receive a list of dozens of IP address, domain names, and MD5 Sums of suspected malware with little or no context, and if you're told a time-frame of the event the window is usually very wide due to the uncertainty that the investigators have in the early stages of an event.

The First Exercise

Before assigning tasks and spinning up the incident response process the first thing that the participants must determine is "who should be informed of the incident," and "who should be involved in the response." Are there people missing from the table that should be participating? Once you've captured your list of people that need to be immediately informed, work out how that notice will be delivered. Do you have after-hours contact information for everyone on that list?

This should be the first deliverable produced from this exercise.

If you have the luxury to hold a multi-day exercise this would be a good time to break. Invite those who were identified as missing from the table, and prepare the emergency contact process documentation. Participants should be taking time to think how they would investigate the limited information provided in the initial notification.

Creating the Time-line of the Attack

As the organizer you know what really happened in the scenario and drive the participants' discovery process. Take the sample time-line below and customize it for your environment. Some changes will be easy to make, others may take a bit of creative and devious thought (e.g. coming up with a plausible way the attackers can establish a command and control channel out of your firm.)

The time-line below is the product of a synthesis of many of the recent details that have been published, stories told to me at bars, and other unreliable and speculative sources. As they say on TV, any similarities to real events is purely coincidental. Efforts have been taken to make the time-line realistic so that there may be value gained from the exercise.

Throughout the time-line I will call out certain check-points for flags as the adversary achieved critical stages in the attack. These flags are presented to organize the discussion and break this large, complex event down into manageable chunks. These can also be used by any future pen-testing process to measure success, or be examined on their own by audit to measure the detection and control efforts in place to defend that flag.

First Phase

First they will collect information about you, your organization and your environment. This will be extremely hard to detect, and you will likely never know when that began or what information they gathered. You can be fairly certain that Google and your own marketing and PR efforts were involved.

There will be some sort of reconnaissance; network scans, or vulnerability scans of your web applications. It will be very difficult to identify what was related to the eventual attack and what was just another day on the Internet.

The first successful attack will likely involve a web application. A SQL injection or file-injection vulnerability will be exploited on one of your web-servers. This is the first flag, a SQL injection leaks out password hashes, or account details. These will be exfiltrated out among HTTP requests from the attack, and any hash values will undergo a cracking process. They have rainbow tables, and parallel-computing resources, so eventually a password will fall to these efforts and they will have a working account on the system. This is another flag: working username and password pair. However, if a remote file-inclusion vulnerability is found and exploited (flag) they now have the opportunity to drop files on the system and execute them via web requests. First it will be a simple file-uploader page that they drop using the file-injection. Then they will upload a web-based control panel on the system (flag), and using the same web access that you provide to the Internet, they can upload and execute any code using the privileges of the web server process ID.

They will drop more tools on the server, targeting the password hashes on the system itself (flag.) These will be pulled off of the system via HTTP and subjected to the same password attacks. One they have working accounts at the system level (flag) they will will use that to log into other systems via the web-based control panel (flag-- can you detect that your web-server is logging into other systems?)

Their target is your Active Domain server (or your firm's equivalent centralized access control system.) Once they have a working account on that server, they will attempt to elevate privileges enough to grab the password hashes. If you use the same admin password on your web servers as you use on your Active Domain server, then they can skip that step-- you made it a lot easier for them.

Second Phase

Armed with working accounts, and user-lists, and other public information about your firm/environment, they will craft targeted emails for key players in your organization. These messages will either contain malicious payloads, or provide links to sites that will compromise the reader and install a remote-access-tool on the system (flag.)

The command and control method used by these tools will be a mixture of novel and old school. Expect something clever like encoded DNS traffic to something as simple as a reverse telnet shell out to a compromised hosting provider. This is where you will need to be creative and customize this for something that would work in your environment. It's also an opportunity to shed some light on any glaring security holes that you're aware of and want upper management to help out with (e.g. A lack of egress filtering, or wide open proxy policy.)

The Heist

At this point, the attackers are in a position to research where your key documents are held and how to grab them. Once they have an internal system under their control, and it's just a matter of time before they either have the Administrator password or the ability to promote an account (flag-- are you alerting on when accounts are reclassified or when administrator is logging in directly to systems?) to administrative privileges. They will move from machine to machine looking for their ultimate target (flag.) Once found, it will go out via existing command and control channels (i.e. The remote access tool or HTTP control panel) or right out via email.

Example Flags for an Exercise

These are the flags I would use for the exercise listed above:

Phase One

• SQL injection discovered on web application
• SQL injection leveraged to expose web application authentication hashes
• Accessed to web application gained using cracked password
• Remote File Inclusion vulnerability discovered in authenticated area of web application
• RFI vulnerability leveraged to execute a control panel
• File uploader script installed on the system via control panel
• Password hash extraction tool uploaded to the web server
• Web-server password hashes extracted
• The web-server attempts to log into the Active Domain controller.

Phase Two

• Member of Upper management receives malicious PDF file
• Member of Sales department receives a malicious spreadsheet
• Member of IT department receives a link to a malicious video
• Remote-Access-Tool is installed on one or more of the spear-phishing targets
• Command-and-Control channel is established to an internal desktop or laptop
• Password extraction tools are installed on the compromised system.
• Password hashes are extracted
• The compromised system logs into the Active Domain controller.
• New accounts are added to the Domain and granted administrator privileges.

 The Heist

• The compromised system mounts the drive containing the crown jewels.
• Files are compressed and uploaded to website on the Internet

Detailing the Flags

For each flag you are going to give an approximate time that it occurs. Starting with the first flag, "SQL injection discovered on web application," note the approximate time elapsed. Some flags will follow in quick succession, others may wait for weeks or months as password hashes are attacked. Phase one and phase two may overlap. You may want to put more flags in to model how the group may explore your environment and try different techniques.

In addition to the timing of each flag. You will want to note how that flag may manifest itself in your organization's logs to justify how the step was discovered in your firm. Not every flag will leave easy log evidence. You might note a flag as being "currently undetectable" by your system, and it would be "discovered" in the exercise from something more intensive like forensic analysis or the discover of a fictitious outside consultant brought in to supplement your team.

Presentation of the Events

After you finish working up your customized time-line of the attack, list out all of the individual flags in chronological order. Come up with a plausible method that you may discover that event, this will guide you in writing the discovery stages of the exercise. For example, given the IP addresses provided by the notifying agency, you might find one of those IP addresses as one of the command and control endpoints and that would lead you to one of the internal infected systems.

It's common to work backwards from the discovery of the incident to the cause. So it makes sense to present the flags in reverse-chronological order-- although there may be a case where discovery of the first and second phases of the attack occur in parallel.

Once you have listed out the order that you present your events prepare some questions surrounding each flag. Initially you will discuss how you would instigate that event, but later you should focus on how you would generically detect an attacker reaching that flag. Another product of this exercise will be a list of recommendations so that you can detect future attackers while they're trying to get in and stop them before they get too far. Nobody wants to get that phone-call telling them that they've had a major compromise.

Wrapping Up?

At the end of this exercise you should have a working CERT contact sheet. Even if they're not initially aware of it, everyone that should be informed of an incident like this is now a member of your organization's Emergency Response Team.

For each flag, you should have listed out your plan to detect when an attacker achieves that stage in your environment. You should have identified existing controls, or point out a need for additional controls. You will likely have identified instances where individual alerts would not warned you of the big picture, but in concert, multiple alerts would have properly highlighted the heist. Followup with a discussion on how you can get your various groups cooperating and sharing log and alert data.

The results of this exercise should be written up, shared with upper management, audit, and your pen-testing team. After that, you'll find that you have a lot more work to do. Sorry. All I can say is that it's better than your CEO getting a phone-call from Special Agent Smith.