Extending Hunting Capabilities in Your Network

Published: 2018-03-23. Last Updated: 2018-03-23 07:30:50 UTC
by Xavier Mertens (Version: 1)

Today's diary is an extension to the one I posted yesterday about hunting for malicious files crossing your network[1]. Searching for new IOCs is nice but there are risks of missing important pieces of information! Indeed, the first recipe could miss some malicious files in the following scenarios:

Case 1: What if a malware is discovered and analysed to extract IOCs but your organization was targeted three weeks ago?
Case 2: What if IOCs are received today but your organization is targeted in the coming one or two months?

Ok, the life of a malware sample (MD5 or SHA1 hash) is very short. They are millions of new malicious files every day. But it’s not the same with IP addresses or domains. I see often malicious IP addresses that are re-used across multiple events in MISP:

Remember, yesterday we exported a list of hashes from the last 30 days from MISP. In some cases, 30 days might already be way too much for some platforms and have to be reduced to fewer days. The scheduled search in Splunk was scanning event from the last hour. If we increase this to events from the last x months or ‘all time’, they are chances to dramatically impact the Splunk performance.

The solve the cases above, let’s create a new tag in MISP called ‘Hunting’ (or whatever you want). All events tagged as ‘Retrohunt’ will have they IOCs exported forever (until the tag is removed):

Let’s generate the list of IOC’s with 2 MISP queries: The last 15 days + events flagged as ‘Hunting’:

wget --header 'Authorization: <redacted>' -O - https://misp/events/hids/md5/download/false/false/false/15d | grep -v "^#") > /tmp/ioc.tmp
wget --header 'Authorization: <redacted>' -O - https://misp/events/hids/md5/download/Hunting | grep -v "^#") >> /tmp/ioc.tmp
(echo md5 && sort -u /tmp/ioc.tmp) > /opt/splunk/etc/apps/search/lookups/malicious_md5.csv

The Splunk lookup table will now contain a sliding window of 15 days with all MD5 hashes and all the hashes flagged as “Hunting”.

To address the case 1describe above, we just need to run a unique big scan once a day at night to search across all the files and the case 2 will be automatically solved because interesting IOCs are now present in the lookup table.

The most important step: How to define which events to tag for ‘Hunting’? Of course, you could generate a list of IOCs based on existing tags or based on organizations that you trust for the quality of their sharings but, in my humble opinion, it's not sufficient. This is a good opportunity to introduce a process to review IOCs. Indeed, the main problem with platforms like MISP (but it’s the same with any tool collecting IOCs) is the flood of IOCs received daily. Keep in mind: The value of an IOC is not only the technical information (the IP address, hash or domain, etc) but also its context. Not all organisations are working in the same business, not all of them have risks to be targeted by known groups. That’s where some threat intelligence is required to define which events received in your MISP are relevant for you and your organization or... not!

[1] https://isc.sans.edu/forums/diary/Automatic+Hunting+for+Malicious+Files+Crossing+your+Network/23473/

Xavier Mertens (@xme)
ISC Handler - Freelance Security Consultant
PGP Key

Keywords:

0 comment(s)

Internet Storm Center

Extending Hunting Capabilities in Your Network

Comments