Using Our API To Adjust iptables Rules

Published: 2017-12-08
Last Updated: 2017-12-08 20:14:07 UTC
by Johannes Ullrich (Version: 1)
6 comment(s)

We are offering a simple (IMHO) API to allow you to script various queries against our databases. One dataset we offer is a list of IP addresses that are scanning the internet for exposed services. The most prominent of these services is likely Shodan. To avoid having any devices from your organization show up in Shodan, you may want to block all scans from known Shodan hosts. We do create a list of these IP addresses and update it daily. The respective API query to retrieve the list is:

https://isc.sans.edu/api/threatlist/shodan/

By default, the list is returned as XML. But it is pretty easy to change the format. All you need to do is add ?json, ?text ... This will make processing with simple scripts rather easy. The "text" format is probably easiest to process with shell tools, but just in case the format is changing later in some subtle way, it is probably safest to use JSON and have the "jq" utility parse it:

curl -s https://isc.sans.edu/api/threatlist/shodan?json | jq '.[] | {ipv4}' | grep ':' | awk '{ print $2 }' | tr -d '"'

This will return a list of all the IP addresses. To use this in iptables, I would recommend setting up a new table. Something like:

iptables -F shodan
iptables -A shodan -j RETURN
for ip in `curl -A "myemailadress" -s https://isc.sans.edu/api/threatlist/shodan?json | jq '.[] | {ipv4}' | grep ':' | awk '{ print $2 }' | tr -d '"'`
do  
  echo $ip
  if [[ $ip =~ ^[0-9\.]+$ ]]
  then
    iptables -A shodan -s $ip -j LOGDROP
  else
    echo "Bad IP Address. Aborting."    
    exit
  fi
done
iptables -D shodan 1

"LOGDROP is a table that will log the packet and drop it. You could also just drop it here, but this would be a bit dangerous as you wouldn't see these dropped packets in your logs which makes debugging problems extra fun.

For a full list of our API functions, see https://isc.sans.edu/api . Please note to use your e-mail address as a user agent. We do not require authentication, but if your script causes issues, then it would be nice if we can check with you vs. just block you.

Of course, test carefully and use at your own risk.

 

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS Technology Institute
STI|Twitter|

 
Keywords:
6 comment(s)

Comments

You can shorten that shell line to:

curl -s https://isc.sans.edu/api/threatlist/shodan?json | jq '.[] | .ipv4' | tr -d '"'
Thanks! that is much better (I probably forgot the . in front of ipv4 when I tried this and it didn't work :( )
60000 iptables rules is pretty inefficient. Placing them into an ipset set would be faster and easier to read from iptables -L.
This works too:

curl -s https://isc.sans.edu/api/threatlist/shodan?text | grep -F ipv4 | grep -oP '\d+\.\d+\.\d+\.\d+'

PS: Would you consider adding a "?list" output format to save us a step?
I second an ipset list.
although i do have to say that if you run the full gambit of all the threat feeds and their ip addresses, the total amount is somewhere around 160K. So it should be noted and recognized that ipset can only hold up to 65535 items in a set. you can however make more than one set xD... anyway

Diary Archives