Threat Level: green Handler on Duty: Russell Eubanks

SANS ISC: Watching the watchers - SANS Internet Storm Center SANS ISC InfoSec Forums


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!
Watching the watchers
A lot of companies today have various IDS and IPS devices implemented in their internal network (especially if you must be compliant with PCI DSS, for example). So these devices get implemented to monitor various traffic at various interfaces/perimeters in a company, but the question I got asked is how can we be sure that the IDS/IPS is doing its job?
 
Obviously, some simple monitoring should be in place – this typically consists of pinging the device or collecting various counters such as memory, CPU usage and similar. This is normally enough to make sure that the device is up and operational. But the question is – how do we make sure that the IDS/IPS is actually detecting malicious traffic or network attacks?
 
An obvious answer to this question is to try to send something malicious and to see if the IDS/IPS correctly identified the attack. So the following options (or all of them) can be implemented:
  • We can automatically download eicar.com and see if the IDS/IPS detected the malicious file.
  • We can perform an automatic scan with nmap or execute any NSE nmap script. Typically, a normal scan (a SYN scan) should trigger the IDS/IPS. This is also a good test to see if the IDS/IPS is detecting network behavior.
  • We can send an exploit over the network. While thinking what to send I browsed through pytbull, which is an IDS/IPS testing framework (more at http://pytbull.sourceforge.net/). Pytbull comes with a bunch of attacks that can be used to test your IDS/IPS installations. At the end I decided that it was too complex, so it was much easier just to take some shellcode, prepend NOP sled to it (yea, so it looks real) and use scapy to send it.
The above "attacks" can now be scheduled – for every schedule, the IDS/IPS device should detect (and/or block) such attacks. To confirm that everything is working ok, it should also generate an appropriate log file which can be then automatically verified with a SIEM; if an attack was executed and there was no detection we know something went wrong with either the IDS/IPS device or the network between the probe and the device – no matter what our standard monitoring is saying.
 
So, what mechanisms do you use to verify your IDS/IPS is working OK? Let us know!

--
Bojan
@bojanz

I will be teaching next: Web App Penetration Testing and Ethical Hacking - SANS Madrid March 2019

Bojan

374 Posts
ISC Handler
After looking at the placement of each IDS/IPS we determine where we need to have the attacker located and what system they would attack. They would send a generic attack for 'GET /attackers_name/etc/passwd' to a web server. This was done during peak hours to ensure that we could find attacks while sustaining max load. The signature should trigger an alert which then triggers a rule in the SIEM. This would not only test the IDS/IPS, but also the SIEM integration. We also noted the time it took from when the attack was conducted to when the alert was generated. We made sure that the alerts were triggered by the IDS, interface and policy that it should have. If there were any issues we worked to resolve them and understand what happened and why.
Apu

1 Posts
We allow our users to have access to the Internet.
Anonymous
Does looking at the logs count? ;-)
John

88 Posts
We usually propose our customers to perform a reactivity-oriented double-blind pentest before whitelisting our source addresses for the subsequent phases.
tillo

7 Posts
I'm looking for solutions to test security devices, especially for Web Application Firewalls (WAF).

Recently, I've found a tool, WAF Walker http://www.waftesting.com/

Do you know any other tool to test web application firewalls?
RafalDarcy

1 Posts
An easy way to validate that your *perimeter* IDS/IPS is working is to run an external vulnerability scan against your public facing network/IP. There are many services that can do this at very reasonable rates. In my organization, we use a professionally hosted external scan engine to scan our perimeters on a daily basis. Not only can we detect new hosts/IPs and/or vulnerabilities on our perimeter, but it validates that our IPS is working as well. When running a scan, we see hundreds of blocked attempts in our IPS logs from the external scan engine's IP address... validating that our IPS is blocking the vulnerability scan packets (which should be considered malicious).
da1212

69 Posts
Another great IDS/IPS testing tool is Evader from StoneSoft: http://evader.mcafee.com/ Also, a good SANS Reading Room GCIA Gold Cert paper "Beating the IPS" https://www.sans.org/reading-room/whitepapers/intrusion/beating-ips-34137 Reblogged at http://ecstaticsec.tumblr.com/
da1212
1 Posts
When events fire, we investigate all the way to the point of pulling the server logs. We've found multiple instances of, say, a SQL injection that the IPS says it blocked, yet the web server's logs clearly indicate the injection attempt came all the way through the defenses.

This has happened with two big-name IPS vendors so far. After trying to work with one of them for years to solve the issue with no traction, we finally removed their appliances. The second vendor we're still working with. They claim they'll fix it, but have taken the stance that the IPS is only supposed to alert to possible issues, and it's the customer's job to investigate. While that may be true to an extent, systems like that are called "Detection" and not "Prevention".
bkendall

7 Posts
bkendall:
Good point. I believe what might be happening is that the IPS is blocking some of the malicious SQL injection packets but not all. I know this happens with our IPS as well because while we see blocked events from a certain IP in the IPS logs, we might still find that IP in the server logs - indicating as you stated that some of the traffic made it through.

What we have done as a countermeasure is to automatically quarantine an IP if it triggers certain filters, such as SQL injection. This way, once there is any hint of malicious traffic from an IP, it gets quarantined and therefore all future traffic from the suspect IP is blocked. However, this has to be configured carefully because you don't want to quarantine valid IPs.
da1212

69 Posts
Quoting Apu:After looking at the placement of each IDS/IPS we determine where we need to have the attacker located and what system they would attack. They would send a generic attack for 'GET /attackers_name/etc/passwd' to a web server. This was done during peak hours to ensure that we could find attacks while sustaining max load. The signature should trigger an alert which then triggers a rule in the SIEM. This would not only test the IDS/IPS, but also the SIEM integration. We also noted the time it took from when the attack was conducted to when the alert was generated. We made sure that the alerts were triggered by the IDS, interface and policy that it should have. If there were any issues we worked to resolve them and understand what happened and why.


Apu, nice idea about measuring time it took to detect the alert. I also like the fact that you test this during peak hours as well.
Bojan

374 Posts
ISC Handler
+1 on bkendall's point - I think it's way too often that people just implement their IPS and forget about it (especially if it was done only to be compliant). With the recent high profile attacks I think we all ought to spend more time verifying that all the whistles and bells we have function properly.
Thanks all for great comments so far :)
Bojan

374 Posts
ISC Handler
Custom signature on IDS/IPS to detect:

- DNS query for "test.domain"
- GET /index.htnl (a custom mis-spell..)

Regards
Anonymous
I gave a look recently on this and among other available tool, you have
- http://testmyids.com/ but more targetting snort rules
- IDS Wake Up http://www.hsc.fr/ressources/outils/idswakeup/index.html.en

After more common tool but more dangerous if you are not in a constrained environment could be using metasploit or download malware...
Julien

10 Posts
All these tools are helpful but it depends how accurately they are working.

www.thedigilibrary.com
Julien
1 Posts

Sign Up for Free or Log In to start participating in the conversation!