Automatic Hunting for Malicious Files Crossing your Network

Published: 2018-03-22
Last Updated: 2018-03-22 07:21:35 UTC
by Xavier Mertens (Version: 1)
0 comment(s)

If classic security controls remain mandatory (antivirus, IDS, etc), it is always useful to increase your capacity to detect suspicious activities occurring in your networks.

Here is a quick recipe that I’m using to detect malicious files crossing my networks. The different components are:
  • MISP[1] - the Malware Information Sharing Platform. I’m running a MISP instance to receive useful IOC’s (Indicator of Compromise) from multiple peers. Common IOCs are IP addresses, domain names, filenames and hashes.
  • Bro[2] is an NSM (Network Security Monitoring) tool that acts like a swiss-army knife on your network. The core feature that will be used here is the extraction of files from network flows. Bro is fully integrated to the SecurityOnion[3] distribution.
  • Splunk - as the orchestrator of the solution.
  • TheHive[4] - A scalable, open source and free Security Incident Response Platform
The first step is to get information about files transferred across your network. If Bro has the capability to store the extracted files in a dump directory (this is very useful for incident response), I prefer to use the ‘files.log’. Indeed, Bro generates multiple log files base on the analysed traffic. On my SecurityOnion instances, I have the following files:
# ls *.log
capture_loss.log   conn.log  dns.log    http_eth1.log    notice.log  smtp.log  software.log  ssl.log    stderr.log  syslog.log  x509.log
communication.log  dhcp.log  files.log  known_hosts.log  sip.log     snmp.log  ssh.log       stats.log  stdout.log  weird.log
Let’s have a look at the ‘files.log’ file:
# grep exe files.log
1521573051.723517        FqfCft31MDKe6sF07k      2606:2800:233:x:x:x:x:x   2a02:a03f:46f2:x:x:x:x:x CKfduySaSsxSrxYu9       HTTP      0       MD5,EXTRACT,PE,SHA1     application/x-dosexec   -       5.348699        F       F       12389248        12389248        0       0       F       -       8e23b0cff15f0ca7bf0ac51a73109a74  26e58f52bc50f79a5a57f1adfaea0ab706bb7f86        -       /nsm/bro/extracted/HTTP-FqfCft31MDKe6sF07k.exe  F       -
You can see interesting fields like the source & destination IP addresses (IPv6 in this case), the file has been transferred via HTTP, has been extracted on disk and there are 2 hashes: MD5 & SHA1. This file is easy to index with Splunk (it can quickly learn the format - details about fields are provided at the beginning of the files and the field separator is <TAB>). Here is the corresponding event indexed by Splunk, let's search for it:
index=securityonion sourcetype=bro_files 8e23b0cff15f0ca7bf0ac51a73109a74
The second step focuses on generating a list of useful IOCs. MISP has an API that helps to extract any kind of information and to format it in your desired output. Let’s extract the MD5 hashes collected for the last 30 days. This is easy to automate with a cron job on your Splunk server:
# crontab -l | grep md5
0 * * * * (echo md5; wget --header 'Authorization: <redacted>' -O - https://misp/events/hids/md5/download/false/false/false/30d | grep -v "^#") >/opt/splunk/etc/apps/search/lookups/malicious_md5.csv
The following file will be created every hour:
# head /opt/splunk/etc/apps/search/lookups/malicious_md5.csv
It is automatically made available in Splunk as a lookup table:
|inputlookup malicious_md5.csv
The final step is to schedule an automatic search at regular intervals in Splunk:
index=securityonion sourcetype=bro_files [| inputlookup malicious_md5.csv]
Any indexed MD5 via files.log and presents in the CSV file will be returned. Finally, let’s create the Splunk alert which will generate alerts in TheHive:
This is a quick example to demonstrate the integration of multiple tools to improve your capacity to detect suspicious activity. The same kind of alerts can be generated for:
  • Connection logs and IP addresses
  • Nameserver resolution and domain names
For sure, there are other ways to get the same results but this is a good example of integrating multiple tools to improve the security posture. Happy hunting!
Xavier Mertens (@xme)
ISC Handler - Freelance Security Consultant
0 comment(s)


Diary Archives