Bots Searching for Keys & Config Files

Published: 2017-07-19
Last Updated: 2017-07-19 06:26:44 UTC
by Xavier Mertens (Version: 1)
4 comment(s)

If you don’t know our "404" project[1], I would definitively recommend having a look at it! The idea is to track HTTP 404 errors returned by your web servers. I like to compare the value of 404 errors found in web sites log files to “dropped” events in firewall logs. They can have a huge value to detect ongoing attacks or attackers performing some reconnaissance. Reviewing 404 errors is one task from my daily hunting-todo-list but it may quickly become unmanageable if you have a lot of websites or popular ones. The idea is to focus on "rare" events that could usually pass below the radar. Here is a Splunk query that I'm using in a daily report:

index=web sourcetype=access_combined status=404
| rex field=uri "(?<new_uri>^\/{1}[a-zA-Z0-9_\-\~]+\.\w+$)"
| cluster showcount=true t=0.6 field=new_uri
| table _time, cluster_count, cluster_label, new_uri | sort cluster_count

What does it do?

  • It searches for 404 errors in all the indexed Apache logs (access_combined)
  • It extracts interesting URI's. I’m only interested in files from the root directory eg. “GET /<name><dot><extension>”
  • It creates “clusters” of common events to help in detecting rare ones.

Here is an example of output (top-20):


Many tested files are basically backup files like I already mentioned in a previous diary[2], nothing changed. But yesterday, I found a bot searching for even more interesting files: configuration files from popular tools and website private keys. Indeed, file transfer tools are used by many webmasters to deploy files on web servers and they could theoretically leave juicy data amongst the HTML files. Here is a short list of what I detected:


Each file was searched with a different combination of lower/upper case characters. Note the presence of ‘rules.abe’ that is used by webmasters to specify specific rules for some web applications[3]. This file could contain references to hidden applications (This is interesting to know for an attacker).

So, keep an eye on your 404 errors and happy hunting!


Xavier Mertens (@xme)
ISC Handler - Freelance Security Consultant

4 comment(s)


Do you ever put "fake" content onto your web-server, e.g., a '/wordpress/' directory, and the '/wordpress/readme.txt' file, and fill that file with a million repeated strings, e.g., 'There is nothing to see here. These are not the droids that you are seeking.' ?

Posting a million obscene words might really annoy the hacker, to the point where they would respond with an all-out attack. Sigh.

But, the point is for them to *NOT* see a '404', and then to respond with further, but obviously fruitless, probing. This wastes their time, and filling their disk-drive with "noise".
No because most of my collected logs are coming from official websites and I don't want to pollute them with fake content.
Note that if you do this, their disk won't be always filled because some bots use HEAD requests instead of GET.
Your comment reminds me of Tom Liston's Labrea.
Good times, good times... :-)
Hah, I see tons of this on my personal webserver still. Countless probes for Wordpress, phpMyAdmin, and other tools. Sometimes I get a 404 for a page and my response is "what the hell are they looking for?" so I have to Google the filename.

It's like spam...lots of output for a little result. But ultimately at no cost to the perpetrators.

Diary Archives