There are some upcoming DShield honeypot [1] changes that introduce some opportunities for additional customization and data analysis. For most users, no additional actions are needed. A couple of those changes:

• dshield.ini file move from /etc/ to /srv/dshield/etc/ - A symbolic link will exist for the previous file location for backward compatibility. If you have automation to update anything in /etc/dshield.ini, you may need to update your scripts. Some tools can recreate the file in the previous location, breaking the link to the new location.
   
• New web honeypot with new options (thanks, Mark Baggett!) - By default, no local logs are generated, which is helpful to save space, but means some customizations may be required if you want to maintain local logs. 

Local Logging

For my own honeypots, I like to maintain local logs. This can be helpful for larger volumes of data or there is a need to analyze the data over time using your own tools. Downloading the information from your ISC portal account is useful, but may not include all data fields. In addition, some daily volumes make it very difficult, if not impossible, to download directly using the options given from the portal. Recently, one of my honeypots had a local web honeypot log of almost 60GB, which was only for one day. This was due to an increase in activity on 5/19/2025, including some URLs noted by Guy a few weeks prior [2]. 
 

Figure 1: Web honeypot logs showing large volumes of traffic from 193.29.13.44 and many other hosts on 5/19/2025. 

 

Figure 2: Volume of data storage for multiple honeypots, showing some large web honeypot logs. 

 

New Web Honeypot [3]

The new web honeypot allows for much more customization for the honeypot itself, which opens up a lot of opportunities to gather data. One of the items that I'm most excited about is that POST data will now be collected within the log files.

Getting POST data from new web honeypot log:

# read all files in /logs starting with "webhoneypot"
# cat /logs/webhoneypot* 

# filter for any data containing the string POST
# grep POST

# find data with the following URL path: "/cgi-bin/../../../../../../../../../../bin/sh"
# jq 'select(.url=="/cgi-bin/../../../../../../../../../../bin/sh")'

# get the POST .data field, sort it, count unique values and sort by the count
# jq .data | sort | uniq -c | sort -n
#

cat /logs/webhoneypot* | grep POST | jq 'select(.url=="/cgi-bin/../../../../../../../../../../bin/sh")' \
| jq .data | sort | uniq -c | sort -n

Figure 3: Gathering POST data from new local web honeypot logs. 

 

For the old web honeypot logs, no data is available in the local logs, but it can be retrieved in many ways if you have PCAPs. One method is tshark [4]. 

# read all PCAP files in the /dumps directory
# for file in /dumps/*.pcap

# for each file, read it with tshark
# do echo "$file";tshark -n -r "$file" 

# filter for POST requests 
# -Y "http.request.method == \"POST\"" 

# select URI and data fields
# -T fields -e http.request.uri -e data.data

# only show results with "cgi-bin" 
# grep "cgi-bin";done

# look for any POST data in files where the URL contains "cgi-bin"
#  cat /logs/webhoneypot* | grep POST | jq 'select(.url | contains("cgi-bin"))' | \
#  jq .data | sort | uniq -c | sort -n


for file in /dumps/*.pcap;do echo "$file";tshark -n -r "$file" -Y "http.request.method == \"POST\"" \
-T fields -e http.request.uri -e data.data | grep "cgi-bin";done

 

Figure 4: POST data exists, but a PCAP is needed. The data does not appear in local JSON logs.

 

Modifications After Honeypot Upgrade

In a previous diary, I went through my steps to customize my honeypot [5] and many of these changes are to try and maintain the same kind of local log data storage. 

Function of Change	File / Folder	Change Made
Add filebeat path to look for log file in new location for forwarding to DShield-SIEM [6]	/etc/filebeat/filebeat.yml	Added to paths: - /srv/log/webhoneypot*.json
Update firewall rules for remote access	/etc/network/iptables	Ran script for automatic update [7]
Add web honeypot local logging	/srv/dshield/etc/dsield.ini	Added to [plugin:tcp:http] stanza: enable_local_logs=true
Add web honeypot local logging location	/srv/dshield/etc/dsield.ini	Added to [plugin:tcp:http] stanza: local_logs_file=/srv/log/webhoneypot-srvconfig.json
Fix dshield.ini permissions due to the use of 'sed -i' inplace editing [8]	/srv/dshield/etc/dshield.ini	sudo chgrp webhpot /srv/dshield/etc/dshield.ini
Update group ownership of folder so 'webhpot' user can save logs to location	/srv/log	sudo chgrp webhpot /srv/log

 

To help automate this a bit, I created a bash script:

# specify file name to modify
file="/etc/network/iptables"

# specify domain of home domain name
domain="isc.sans.edu"

# delete any rule specifying destination port 12222
sudo sed -i "/\b\(dport 12222\)\b/d" $file

# specify ip addresses to allow for admin access
# space delimited
custom_ips=""
private_ips="172.16.0.0/12 192.168.0.0/16 10.0.0.0/8"

# get primary interface to the internet
interface=$(ip route get 1.1.1.1 | grep -Po '(?<=dev\s)\w+' | cut -f1 -d ' ')

# get remote IP address of my home domain
# only use first result
remoteip=$(host $domain | grep "has address" | cut -d " " -f 4 | head -1)

# add rule after line 'START: allow access to admin ports for remote IPs'
# double quotes used to expand variables while preserving whitespace
sudo sed -i "/START: allow access to admin ports for local IPs/a -A INPUT -i $interface -s $remoteip -p tcp --dport 12222 -j ACCEPT" $file

# add any other ip addresses you may want
# add rule after line 'START: allow access to admin ports for custom IPs'
# double quotes used to expand variables while preserving whitespace
for item in $custom_ips; do
  sudo sed -i "/START: allow access to admin ports for local IPs/a -A INPUT -i $interface -s $item -p tcp --dport 12222 -j ACCEPT" $file
done

# add any other ip addresses you may want
# add rule after line 'START: allow access to admin ports for private IPs'
# double quotes used to expand variables while preserving whitespace
for item in $private_ips; do
  sudo sed -i "/START: allow access to admin ports for local IPs/a -A INPUT -i $interface -s $item -p tcp --dport 12222 -j ACCEPT" $file
done

# add extra logging location for longer retention of iptables logs
sed -i '/localcopy\=/d' /srv/dshield/etc/dshield.ini
sed -i '/\[plugin:tcp:http\]/i localcopy=/logs/dshield_firewall_.log' /srv/dshield/etc/dshield.ini

# update new dshield.ini to enable local web-honeypot logging
sed -i '/enable_local_logs\=/d' /srv/dshield/etc/dshield.ini
sed -i '/\[plugin\:tcp\:http\]/a enable_local_logs\=true' /srv/dshield/etc/dshield.ini
sed -i '/local_logs_file\=/d' /srv/dshield/etc/dshield.ini
sed -i '/\[plugin\:tcp\:http\]/a local_logs_file\=\/srv\/log\/webhoneypot-srvconfig.json' /srv/dshield/etc/dshield.ini

# fix group ownership permissions
sudo chgrp webhpot /srv/dshield/etc/dshield.ini

# modify permission for folder I'm putting my logs in so web honeypot can write to it
sudo chgrp webhpot /srv/log

# add new file logging location to filebeat for web honeypot logs: /srv/log/webhoneypot*.json
sudo sed -i '/\/srv\/log\/webhoneypot\*.json/d' /etc/filebeat/filebeat.yml
sudo sed -i '/srv\/db\/webhoneypot\*.json/a \ \ \ \ \-\ \/srv\/log\/webhoneypot\*.json' /etc/filebeat/filebeat.yml

# restart web honeypot after changes
sudo systemctl restart web-honeypot

 

This automated my reconfiguration, but without doing anything else, the web honeypot log file would grow forever. Since I still like having my honeypot files broken down by day, I created a cron job to move the web honeypot file to my log archive location every day after it reboots. 

# -------------script contents-------------
# get the current date and time to be used within web-honeypot filename
TIMESTAMP=`date "+%Y-%m-%d_%H:%M:%S"`

mv /srv/log/webhoneypot-etcconfig.json /logs/webhoneypot-etc-$TIMESTAMP.json
mv /srv/log/webhoneypot-srvconfig.json /logs/webhoneypot-srv-$TIMESTAMP.json

# -------------cron contents-------------
@reboot sleep 30 && ~/move_webhoneypot_logs.sh

 

During the process I learned that command line tools can be useful, but may create some challenges when not properly understood, such as "sed -i". Thoughts or suggestions for the future? Let us know and keep logging!

 

[1] https://github.com/DShield-ISC/dshield
[2] https://isc.sans.edu/diary/31906
[3] https://github.com/DShield-ISC/dshield/tree/dev/srv/web
[4] https://www.wireshark.org/docs/man-pages/tshark.html
[5] https://isc.sans.edu/diary/30024
[6] https://github.com/bruneaug/DShield-SIEM
[7] https://isc.sans.edu/diary/Honeypot+Iptables+Maintenance+and+DShieldSIEM+Logging/31876
[8] https://unix.stackexchange.com/questions/276651/sed-with-inplace-editing-changes-group-ownership-of-file

 

--
Jesse La Grew
Handler