Threat Level: green Handler on Duty: Brad Duncan

SANS ISC: Using Our API To Adjust iptables Rules - SANS Internet Storm Center SANS ISC InfoSec Forums


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!
Using Our API To Adjust iptables Rules

We are offering a simple (IMHO) API to allow you to script various queries against our databases. One dataset we offer is a list of IP addresses that are scanning the internet for exposed services. The most prominent of these services is likely Shodan. To avoid having any devices from your organization show up in Shodan, you may want to block all scans from known Shodan hosts. We do create a list of these IP addresses and update it daily. The respective API query to retrieve the list is:

https://isc.sans.edu/api/threatlist/shodan/

By default, the list is returned as XML. But it is pretty easy to change the format. All you need to do is add ?json, ?text ... This will make processing with simple scripts rather easy. The "text" format is probably easiest to process with shell tools, but just in case the format is changing later in some subtle way, it is probably safest to use JSON and have the "jq" utility parse it:

curl -s https://isc.sans.edu/api/threatlist/shodan?json | jq '.[] | {ipv4}' | grep ':' | awk '{ print $2 }' | tr -d '"'

This will return a list of all the IP addresses. To use this in iptables, I would recommend setting up a new table. Something like:

iptables -F shodan
iptables -A shodan -j RETURN
for ip in `curl -A "myemailadress" -s https://isc.sans.edu/api/threatlist/shodan?json | jq '.[] | {ipv4}' | grep ':' | awk '{ print $2 }' | tr -d '"'`
do  
  echo $ip
  if [[ $ip =~ ^[0-9\.]+$ ]]
  then
    iptables -A shodan -s $ip -j LOGDROP
  else
    echo "Bad IP Address. Aborting."    
    exit
  fi
done
iptables -D shodan 1

"LOGDROP is a table that will log the packet and drop it. You could also just drop it here, but this would be a bit dangerous as you wouldn't see these dropped packets in your logs which makes debugging problems extra fun.

For a full list of our API functions, see https://isc.sans.edu/api . Please note to use your e-mail address as a user agent. We do not require authentication, but if your script causes issues, then it would be nice if we can check with you vs. just block you.

Of course, test carefully and use at your own risk.

 

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS Technology Institute
STI|Twitter|

 
Defending Web Applications Security Essentials - SANS Amsterdam September 2018

Johannes

3318 Posts
ISC Handler
You can shorten that shell line to:

curl -s https://isc.sans.edu/api/threatlist/shodan?json | jq '.[] | .ipv4' | tr -d '"'
Athanasius

7 Posts
Thanks! that is much better (I probably forgot the . in front of ipv4 when I tried this and it didn't work :( )
Johannes

3318 Posts
ISC Handler
60000 iptables rules is pretty inefficient. Placing them into an ipset set would be faster and easier to read from iptables -L.
Anonymous
This works too:

curl -s https://isc.sans.edu/api/threatlist/shodan?text | grep -F ipv4 | grep -oP '\d+\.\d+\.\d+\.\d+'

PS: Would you consider adding a "?list" output format to save us a step?
Mike Donovan

4 Posts
I second an ipset list.
jACKtheRipper

55 Posts
although i do have to say that if you run the full gambit of all the threat feeds and their ip addresses, the total amount is somewhere around 160K. So it should be noted and recognized that ipset can only hold up to 65535 items in a set. you can however make more than one set xD... anyway
jACKtheRipper

55 Posts

Sign Up for Free or Log In to start participating in the conversation!