Analyzing outgoing network traffic

Published: 2012-08-23
Last Updated: 2012-08-23 07:25:41 UTC
by Bojan Zdrnja (Version: 1)
11 comment(s)

We all know that network traffic contains real treasure when trying to identify malicious activities. Various organizations recognized this and even mandate that IDS or IPS systems are implemented.

However, such systems typically have similar problems as anti-virus products – they depend either on pre-made signatures or some kind of heuristics which can be (sometimes easily) evaded.

At the same time, in the AV world we can see that more vendors rely on things such as cloud scanning and reputation systems.

One of the things I often recommend to people is that they check outgoing network sessions created by their networks – not only established connections but also various attempts. For example, you should regularly monitor your firewall logs to see what traffic has been dropped – but put more effort into analyzing what egress connections were blocked since that can help you identify potentially infected (or hacked) machines on your network.

The best example of when such analysis really pays off is RSA Security – through egress log analysis they found out that the hacker that compromised their network used FTP to transfer files to an external machine. This should make you ask yourself – do you monitor egress connections to detect big(ger) transfers to external hosts, especially those in weird locations?

Another thing that I found really useful is to correlate those connection attempts to known bad reputation sources; this is where we get to the beginning of this diary. Such correlation can really add value to your firewall/router data – knowing that an internal IP address tried to connect to an external IP address, and that this connection attempt was blocked is good, but knowing that the external IP address is actually a ZeuS C&C really adds value!

Some of the reputation sources that are free, and that I found to be working really well are the following (in no particular order):

Do you use other reputation sources? Anything you wish to add to this list? Let us know!

--
Bojan
INFIGO IS

11 comment(s)

Comments

For me, this one is good too... http://www.malwaredomainlist.com/mdl.php

Sorry to ask, but how reliable is RBN list for you guys?
We use those three abuse.ch lists as well and also the the lists of the spamhaus project (www.spamhaus.org). Apparently the spamhaus project has started integrating the abuse.ch lists ("Spamhaus Botnet C&C List"), for more information -> http://www.spamhaus.org/bgpf/. But we have just recently started investigating correlations between IDS-Positives and reputation sources. So we don't have much experience gathered, but it looks promising. But I wonder how reliable the RBN list actually is, we don't use that one,

But since many years at our institute (Swiss Federal Institute of Technology) we analyze the egress connections. We use statistical methods to automatically detect peaks and also correlations between positives. The last powerpoint presentation of our implementation can be found here:
https://www1.ethz.ch/id/services/list/security/workshops/IDS-ETHS-SWITCH_SecWG2011
It is not quite up to date some, things are missing there, but essentially it is a correct description of our setup.

Christian Hallqvist
www.ethz.ch
Might be of use, some overlap, inbound and outbound.

http://rules.emergingthreats.net/open/suricata/rules/compromised-ips.txt
http://rules.emergingthreats.net/open/suricata/rules/botcc.rules
http://rules.emergingthreats.net/open/suricata/rules/rbn-ips.txt
https://www.projecthoneypot.org/list_of_ips.php
http://rules.emergingthreats.net/open/suricata/rules/tor.rules
http://rules.emergingthreats.net/open/suricata/rules/compromised.rules
http://www.malwaredomainlist.com/hostslist/ip.txt
http://rules.emergingthreats.net/open/suricata/rules/rbn.rules
http://www.mtc.sri.com/live_data/attackers/
http://intel.martincyber.com/ip/
https://spyeyetracker.abuse.ch/blocklist.php?download=ipblocklist
https://zeustracker.abuse.ch/blocklist.php?download=ipblocklist
https://reputation.alienvault.com/reputation.generic
https://www.openbl.org/lists/base.txt
http://www.blocklist.de/lists/ssh.txt
https://palevotracker.abuse.ch/
http://www.malwaregroup.com/ipaddresses
http://www.ciarmy.com/list/ci-badguys.txt
http://www.malware.com.br/cgi/submit?action=list
The complement to the listed would include
-monitoring outbound routes (may be an infrastructure activity if firewall is not adequately monitoring outbound)
- monitor, control and log any outbound encryption (dont assume all encryption is good_)
-special attention to open proxies (e.g. hidemyass.com )
http://www.autoshun.org/files/shunlist.html

Thanks for all comments so far! I will collect everything we receive from our readers and will update a diary in a day or two.
For what it's worth (probably not much), I downloaded all the lists mentioned above, pulled out just the IPs. There were 18,796 uniq IPs or CIDRs (didn't eliminate given IPs contained in given CIDRs) from 27 pages (didn't follow links on pages to other lists). I georeference the addresses, and here are the top 12 countries:

Rank #Addrs CC
---- ------ --
1 5898 US
2 1819 CN
3 1435 RU
4 1180 DE
5 736 NL
6 510 UA
7 463 GB
8 424 FR
9 406 KR
10 380 CA
11 330 BR
12 307 TW

I'm a little surprised to see Germany at #4, because it doesn't seem to show up as much in the contexts with which I usually deal.

Christian mentions Spamhaus. He points to the bgpf services that they ask a fee for.
Free lists from Spamhaus are DROP en EDROP:
http://www.spamhaus.org/drop/
While IP reputation is useful, unless you are able to see some detail of the communication (e.g. HTTP URLs accessed), due to web sites being hosted on the same IP as harmless web sites you cannot fully be sure the communication is suspect. Egress filtering along with DNS inspection (or DNS sinkhole) offers much greater insight, obviously if backed up with IDS/IPS filtering to give defence in depth (or inspection in depth if passive). I am not saying IP reputation is not a useful layer of defence, just worth remembering it is only one layer and not an entire solution.
I've always found that authenticating outbound traffic as much as possible through active directory an effective method of limiting the damage of malware that runs in the system context. If the malware attempts to egress on commonly open ports such as 80/443, without proper AD creds, the traffic goes nowhere and gets logged by your gateway device.

Diary Archives