Threat Level: green Handler on Duty: Xavier Mertens

SANS ISC: Internet Storm Center - SANS Internet Storm Center Internet Storm Center

Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!

Latest Diaries

Automatic Hunting for Malicious Files Crossing your Network

Published: 2018-03-22
Last Updated: 2018-03-22 07:21:35 UTC
by Xavier Mertens (Version: 1)
1 comment(s)

If classic security controls remain mandatory (antivirus, IDS, etc), it is always useful to increase your capacity to detect suspicious activities occurring in your networks.

Here is a quick recipe that I’m using to detect malicious files crossing my networks. The different components are:
  • MISP[1] - the Malware Information Sharing Platform. I’m running a MISP instance to receive useful IOC’s (Indicator of Compromise) from multiple peers. Common IOCs are IP addresses, domain names, filenames and hashes.
  • Bro[2] is an NSM (Network Security Monitoring) tool that acts like a swiss-army knife on your network. The core feature that will be used here is the extraction of files from network flows. Bro is fully integrated to the SecurityOnion[3] distribution.
  • Splunk - as the orchestrator of the solution.
  • TheHive[4] - A scalable, open source and free Security Incident Response Platform
The first step is to get information about files transferred across your network. If Bro has the capability to store the extracted files in a dump directory (this is very useful for incident response), I prefer to use the ‘files.log’. Indeed, Bro generates multiple log files base on the analysed traffic. On my SecurityOnion instances, I have the following files:
# ls *.log
capture_loss.log   conn.log  dns.log    http_eth1.log    notice.log  smtp.log  software.log  ssl.log    stderr.log  syslog.log  x509.log
communication.log  dhcp.log  files.log  known_hosts.log  sip.log     snmp.log  ssh.log       stats.log  stdout.log  weird.log
Let’s have a look at the ‘files.log’ file:
# grep exe files.log
1521573051.723517        FqfCft31MDKe6sF07k      2606:2800:233:x:x:x:x:x   2a02:a03f:46f2:x:x:x:x:x CKfduySaSsxSrxYu9       HTTP      0       MD5,EXTRACT,PE,SHA1     application/x-dosexec   -       5.348699        F       F       12389248        12389248        0       0       F       -       8e23b0cff15f0ca7bf0ac51a73109a74  26e58f52bc50f79a5a57f1adfaea0ab706bb7f86        -       /nsm/bro/extracted/HTTP-FqfCft31MDKe6sF07k.exe  F       -
You can see interesting fields like the source & destination IP addresses (IPv6 in this case), the file has been transferred via HTTP, has been extracted on disk and there are 2 hashes: MD5 & SHA1. This file is easy to index with Splunk (it can quickly learn the format - details about fields are provided at the beginning of the files and the field separator is <TAB>). Here is the corresponding event indexed by Splunk, let's search for it:
index=securityonion sourcetype=bro_files 8e23b0cff15f0ca7bf0ac51a73109a74
The second step focuses on generating a list of useful IOCs. MISP has an API that helps to extract any kind of information and to format it in your desired output. Let’s extract the MD5 hashes collected for the last 30 days. This is easy to automate with a cron job on your Splunk server:
# crontab -l | grep md5
0 * * * * (echo md5; wget --header 'Authorization: <redacted>' -O - https://misp/events/hids/md5/download/false/false/false/30d | grep -v "^#") >/opt/splunk/etc/apps/search/lookups/malicious_md5.csv
The following file will be created every hour:
# head /opt/splunk/etc/apps/search/lookups/malicious_md5.csv
It is automatically made available in Splunk as a lookup table:
|inputlookup malicious_md5.csv
The final step is to schedule an automatic search at regular intervals in Splunk:
index=securityonion sourcetype=bro_files [| inputlookup malicious_md5.csv]
Any indexed MD5 via files.log and presents in the CSV file will be returned. Finally, let’s create the Splunk alert which will generate alerts in TheHive:
This is a quick example to demonstrate the integration of multiple tools to improve your capacity to detect suspicious activity. The same kind of alerts can be generated for:
  • Connection logs and IP addresses
  • Nameserver resolution and domain names
For sure, there are other ways to get the same results but this is a good example of integrating multiple tools to improve the security posture. Happy hunting!
Xavier Mertens (@xme)
ISC Handler - Freelance Security Consultant
1 comment(s)

If you have more information or corrections regarding our diary, please share.

Recent Diaries

Surge in blackmailing?
Mar 21st 2018
1 day ago by Xme (4 comments)

Administrator's Password Bad Practice
Mar 20th 2018
2 days ago by Xme (1 comment)

Wireshark and USB
Mar 17th 2018
5 days ago by DidierStevens (3 comments)

[Wireshark-announce] Wireshark 2.5.1 is now available
Mar 16th 2018
6 days ago by Basil (0 comments)

VMWARE Security Advisory: VMSA-2018-0008
Mar 16th 2018
6 days ago by Basil (0 comments)

View All Diaries →

Latest Discussions

Splunk: Any way to fetch logs via ssh
created Mar 15th 2018
1 week ago by Anonymous (1 reply)

Possible new worm activity
created Mar 13th 2018
1 week ago by Anonymous (0 replies)

Detecting the memcached issue
created Mar 9th 2018
1 week ago by David (0 replies)

OSINT tools and links
created Mar 9th 2018
1 week ago by Anonymous (0 replies)

IPhone VPN connection error.
created Mar 7th 2018
2 weeks ago by Janecollen (0 replies)

View All Forums →

Latest News

View All News →

Top Diaries

Wide-scale Petya variant ransomware attack noted
Jun 27th 2017
8 months ago by Brad (6 comments)

Using a Raspberry Pi honeypot to contribute data to DShield/ISC
Aug 3rd 2017
7 months ago by Johannes (16 comments)

Detection Lab: Visibility & Introspection for Defenders
Dec 15th 2017
3 months ago by Russ McRee (2 comments)

Second Google Chrome Extension Banker Malware in Two Weeks
Aug 29th 2017
6 months ago by Renato (0 comments)

Maldoc with auto-updated link
Aug 17th 2017
7 months ago by Xme (2 comments)