Incident Response vs. Incident Handling

Published: 2009-04-16
Last Updated: 2011-01-25 00:03:23 UTC
by Adrien de Beaupre (Version: 1)
7 comment(s)

One of the things that comes ups frequently in discussion is the difference between incident response, and incident handling.  Anyone who has had to deal with an incident has likely encountered this situation at least once.  While attempting to work on analyzing what the #$%& happened you always have your senior executive and/or clients hanging over your shoulder constantly bugging you for more details. The containment plan needs to be worked out, someone needs to liaise with Legal/HR/PR, management wants an update, the technical staff need direction or assistance, teams need to be coordinated, everyone wants to be in the loop, lots of yelling is going on, external IRTs want to know why your network is attacking theirs, nobody can locate the backups, keeping track of activities, taking notes, and the list goes on… No wonder people regularly burn out during incidents! Incident handling is obviously not a solo sport.

The best line for this problem came from the hot wash (post incident debrief) of a major exercise, where one of the leads stood up to the table of senior executives, and calmly explained "No, no you will NOT go talking to the team.  You will talk to me. Only me.  It is my job to keep you off their backs so they can do actual work".

That is the difference between Incident Response, and Incident Handling.  Incident Response is all of the technical components required in order to analyze and contain an incident.  Incident Handling is the logistics, communications, coordination, and planning functions needed in order to resolve an incident in a calm and efficient manner. Yes, there are people who can fulfill either role, but typically not at the same time. The worse things get, the greater the requirement for the two different roles becomes.

These two functions are best performed by at least two different people, although in larger scale incidents this may need to be two leads who work closely together to coordinate the activities of two separate teams. In smaller environments ideally the Incident team should still always be two people, one responding, and the other taking notes and communicating with stakeholders.

It should also be noted that the skill sets for these two are different as well.  Incident Response requires strong networking, log analysis, and forensics skills; incident handling strong communications and project management skills.  These are complementary roles which allow the responders to respond, the team to work in a planned (or at least organized chaos) fashion and the rest of the world to feel that they have enough information to leave the team alone to work.

Thanks to a former colleague who helped me writing this.

Thoughts or feedback?

Update: Rex wrote in the following: US workers in emergency services (fire, police, ambulance, first aid), are trained in the Incident Command System:  http://en.wikipedia.org/wiki/Incident_Command_System
which defines a number of important roles in handling an emergency incident.

Small teams won't have enough members to put one person in each role.  Experience shows that the "hands-on" members *must be left alone*, even if you need to do on-the-spot recruitment to handle PR, crowd control, logistics, etc.  Another key concept is "one person in charge per function" - one commander, one PR person, one chief medic, etc.

Maybe the IT security incident response/handling world should steal good ideas from people who deal with life-and-death emergencies.  Soon, IT emergencies will be life-and-death emergencies.

Cheers,
Adrien de Beaupré
Intru-shun.ca Inc.

 

7 comment(s)

Comments

Great diary post; I've seen the terms get switched out in discussion many times, indicating there is some confusion on definition. It's good to see someone put them into context.
The term we used in our incident response plan is \"management liason\". Basically someone who would communicate the state of the ongoing incident response to upper management, partners, etc, and act as a buffer between the people doing the technical work and anyone who would like to come and bug them. Depending on the severity of the incident of course, this person/role may or may not be required.
Check out the free training resources for the federal ICS here: http://training.fema.gov/IS/crslist.asp
As a former Lifeguard at a large aquatic park and a current Information Assurance major, I could not agree more that the two fields align very closely. Emergency Action Plans apply in both situations, and both must be trained constantly in order for the team to respond correctly without thinking. There is no time for thinking - there is only time for action and response. This can ONLY come after training and more training. I can still recite to you every step of our pool's EAP and run you through every step of recognize/response that was required by the American Red Cross, as well as any job of any of my teammates. Head/neck/back injury off a water slide into 4 feet of water? No problem. Lost child? No problem. We trained for it at least once a week and at least once every summer, the ambulance was called to take someone to the hospital. So far (knock on wood), partially due to our response time and steady training, no one has died at our pool.

We really need some standards for this in the IT security world. I've seen types of EAP, but there seems to be little consistency between the plans and even less people who know the plans even exist.
Rather than waiting until a post mortum on an IR exercise, document the interface process in your approved incident response plan. In this context it should be specified by role. We designate a primary and a secondary for the role (this would be a technical manager).

Another point to consider carefully is whether Legal should be part of the CSIRT or not. Our experience has led us to include legal for the following reasons:

1) Privilage. There may be times when a technical discussion may have legal ramifactions. By having it with counsel this may be considered a privilaged conversation (think lawsuits).

2) There may need to be decisions as to whether the incident will involve providing evidence for a criminal investigation (or civil attempts to recover damages) or whether the top priority is recovery of services. Having legal part of the team can be very useful.

For the larger handling issues we have a seperate Crisis Management Team.
TaoSecurity's Response to this entry: http://taosecurity.blogspot.com/2009/04/speaking-of-incident-response.html
TaoSecurity's Response to this entry: http://taosecurity.blogspot.com/2009/04/speaking-of-incident-response.html

Diary Archives