ISC Snapshot: SpectX IP Hitcount Query

Published: 2020-06-30
Last Updated: 2020-06-30 05:53:07 UTC
by Russ McRee (Version: 1)
1 comment(s)

SpectX was the subject of an ISC post on SpectX4DFIR back in late April. Raido from SpectX provides us with a query to count hits from IPs during different time intervals.

This can be one way of detecting possible bots and automated queries. Running the query below will tell you:

  1. On how many different days do we have hits from a particular IP (column ‘days’)?
  2. On how many different days did we see this IP every hour, 24 hours in a row (column ‘full_days’)?
  3. During how many different hours did we get hits from this IP? (column ‘hours’)?

I ran the query below, slightly modified from Raido’s original, against the April 2020 log file for You can run this on any log file that contains timestamps and IP-addresses, just change the path, pattern and field names accordingly.

| parse(pattern:$[/user/patterns/apache/apacheLog.sxp])
| select(hour:timestamp[1 hour], clientIp) 
| group(hour, clientIp) 
| select(day:hour[24 hour], clientIp, hours:count(*))
| group(day, clientIp)
| select(clientIp, days:count(*), full_days:count(hours = 24), hours:sum(hours))
| group(clientIp)
| sort(full_days desc)

The results as seen in Figure 1 provided immediate insights.

Figure 1: SpectX IP hitcount query result

As promised, these IPs as noted in the results per Figure 1 are all making constant calls to my site, all day, every day. Each are calling my index.xml file, some appear to be RSS readers or scrapers, which is fairly routine. Seems like a lot of needless connect and compute cycles for a low traffic, static site such as mine.
Some of these IPs are definitely of ill repute however., originating out of Nuremberg, Bavaria scored a near perfect 99 of 100 for fraudulent behavior and malicious activity based on recent actions according to IPQSFigure 2 bears this out.

Figure 2: IPQS declares badness

This is useful little query to quickly detect possible bots and automated queries. Hopefully you’ve already downloaded SpectX and given a try after a last post. Load it back up and feed a log. If you want a copy of the log as utilized for this post, let me know via socials or email.

Cheers…until next time.

Russ McRee | @holisticinfosec

1 comment(s)


Thanks for the post, Russ!
One more tip - you can enrich the IPs with MaxMind's GeoLite to get more instant info on the (weird) IPs.
1) Create a Maxmind account and configure accessing the data in SpectX:
2) Run the IP-query in this post
3) Right-click on one of the IPs in the resultset > Add Code Snippet > lookup Maxmind.
The following select command is added to the query:
| select(ip, cc(ip), asname(ip), geo(ip), *)
4) Click Run and continue querying the new fields :)

Diary Archives