All I want this Tuesday: More Data

Published: 2020-07-28
Last Updated: 2020-07-28 18:02:17 UTC
by Johannes Ullrich (Version: 1)
0 comment(s)

We are making more and more data available via our API.  Couple things just added:

  • A combined "Threatintel" feed that includes basic categories of notable IPs
  • A list of subnets used by prominent cloud providers

First a few reminders about our API:

You may use data from our API for free. This includes the use in commercial networks. We only ask you to talk to us about licensing if you are reselling the data as part of a product. For example, if you include this data in your own data feeds and you are charging money for the feed.

At this point, we are not asking for any kind of authentication. We are doing minimal tracking and do not care who is downloading the data. However, we may block users who we feel abuse the data. We will try to contact you if you include contact information in your user agent and it is possible that we will block certain "generic" user agents. Best to customize the user agent somehow.

What I do ask for in exchange for using these feeds:

  • provide feedback. Let us know how you use these feeds, and please let us know about problems (see our contact page)
  • Consider contributing data via our honeypot.

The use case I am envisioning for the data is to include it in a SIEM or other log monitoring products to add "color" to the IP address. It may be useful to know that the IP attacking you is just "yet another infected bot".

WE DO NOT RECOMMEND THE USE OF OUR DATA AS A SIMPLE BLOCK LIST

Our data includes false positives. I see it as a feature as false positives is also something we continuously learn from. For example when it comes to DoS attacks, or artifacts of firewalls blocking "odd" packets. Best case: If you are blocking based on our "Top 100" list, you are blocking a bunch of bots that scan for vulnerabilities you are hopefully not susceptible to. And if you are vulnerable to any of them: There are bot number 101-123183849 that will still get you.

Our API offers data in different formats. Just add the format specifier to the URL. For example "?json" for JSON which is probably now the preferred output format.

Back to the new datafeeds I added:

"Intelfeed" https://isc.sans.edu/api/intelfeed

A lot of organizations like to ingest random feeds of "threatintel" data. This feed is trying to extract some notable data from across our different collections. It includes for port scanners detected by out DShield sensors, hosts scanning for web vulnerabilities and ssh brute force bots reported by our honeypots and data from various other feeds we are collection. A quick snippet:

   {
    "ip": "1.119.147.51",
    "description": "DShield Ports: 65529,16379,6379,22,1433"
  },
  {
    "ip": "1.119.195.58",
    "description": "dshieldssh"
  },
  {
    "ip": "1.160.6.79",
    "description": "talos"
  },
  {
    "ip": "5.11.11.10",
    "description": "tldns"
  },

  • The first IP is a host scanning various ports based on our DShield data (I only include hosts that hit several target IPs to limit the size of the feed).
  • The second IP scans for SSH servers
  • The third IP is included in the Talos IP blocklist
  • Finally, 5.11.11.10 is a name server for a top level domain (don't block these! ;-) )

The number of categories is likely going to increase.

Cloud IPs https://isc.sans.edu/api/cloudips

This is a simple feed including prefixes used by major cloud providers. Right now, it includes AWS, Azure, Google and Oracle with more to come. This one is pretty straight forward.

These feeds are only as good as you make them. Feedback is very welcome. Please use our Contact Page for feedback.

 

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS Technology Institute
Twitter|

0 comment(s)

Comments


Diary Archives