DShield Sensor JSON Log to Elasticsearch
My current project has been to rebuild my home DShield sensor from a Rasberry Pi to a Ubuntu 20.04.5 LTS server to be able to process my sensor logs into Elasticsearh. I use as a guide the example listed here (my ELK is version 8.x) sending the cowrie.json logs to a remote ELK server (version 8.4.1) using Filebeat and Logstash. However, my steps were a little different than the reference:
1 - Install the OS (basic server version)
2 - Add the following packages
$ sudo apt-get install net-tools open-vm-tools htop ntp bind9-utils vim network-manager
$ sudo systemctl start NetworkManager
$ sudo systemctl enable NetworkManager
$ sudo nmcli device show
Configure sensor static IP [2]
$ sudo vi /etc/netplan/00-installer-config.yaml
3 - Install DShield sensor using the steps and script shared in Github
4 - Install and configure filebeat
$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
$ echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
$ sudo apt-get update && sudo apt-get install filebeat
→ Edit and configure the filebeat.yml with the following setting to send the logs to Elasticsearch
$ sudo vi or nano /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
json.keys_under_root: true
json.add_error_key: true
json.message_key: log
# Unique ID among all inputs, an ID is required.
id: cowrie
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /srv/cowrie/var/log/cowrie/cowrie.json*
output.logstash:
hosts: ["192.168.xx.xx:5044"]
$ sudo systemctl start filebeat
$ sudo systemctl status filebeat
$ sudo systemctl enable filebeat
5 - Import the Cowrie mapping template into Elasticsearch via Dev Tools.
6 - Copy and configure Logstash parser (i.e. ELK destination, certificates, etc)
I have shared the Kibana Cowrie mapping template here and logstash parser here.
$ sudo systemctl restart logstash
After a logstash service restart, monitor the service to ensure there are no errors and a file like this, cowrie-2023.01.21-000001 should be visible in the Index Management in Kibana. The shared dashboard available here to be imported in Kibana in the shared object (version => 8.4.1).
Kibana Cowrie logs
Kibana Dashboard
The activity shown in this dashboard should be the same as the logs the sensor is reporting to DShield. Now this information is available to compare against threatintel.
DShield Sensor Log Location
Log files: /srv/cowrie/var/log/cowrie/
Uploaded files: /srv/cowrie/var/lib/cowrie/downloads
Firewall logs: /var/log/dshield.log
Weblogs: /srv/www/DB/swebserver.sqlite
[1] https://cowrie.readthedocs.io/en/latest/elk/README.html
[2] https://www.serverlab.ca/tutorials/linux/administration-linux/how-to-configure-networking-in-ubuntu-20-04-with-netplan/
[3] https://isc.sans.edu/tools/honeypot/
[4] https://isc.sans.edu/diary/29412
[5] https://isc.sans.edu/diary/29370
[6] https://isc.sans.edu/diary/28872
[7] https://handlers.sans.edu/gbruneau/elk/logstash-filter-cowrie.conf
[8] https://handlers.sans.edu/gbruneau/elk/cowrie_8.4.1.ndjson
[9] https://handlers.sans.edu/gbruneau/elk/cowrie-mapping-template.txt
-----------
Guy Bruneau IPSS Inc.
My Handler Page
Twitter: GuyBruneau
gbruneau at isc dot sans dot edu
Comments
Anonymous
Dec 3rd 2022
9 months ago
Anonymous
Dec 3rd 2022
9 months ago
<a hreaf="https://technolytical.com/">the social network</a> is described as follows because they respect your privacy and keep your data secure. The social networks are not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go.
<a hreaf="https://technolytical.com/">the social network</a> is not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go. The social networks only collect the minimum amount of information required for the service that they provide. Your personal information is kept private, and is never shared with other companies without your permission
Anonymous
Dec 26th 2022
9 months ago
Anonymous
Dec 26th 2022
9 months ago
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> nearest public toilet to me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
Anonymous
Dec 26th 2022
9 months ago
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> nearest public toilet to me</a>
<a hreaf="https://defineprogramming.com/the-public-bathroom-near-me-find-nearest-public-toilet/"> public bathroom near me</a>
Anonymous
Dec 26th 2022
9 months ago
Anonymous
Dec 26th 2022
9 months ago
https://defineprogramming.com/
Dec 26th 2022
9 months ago
distribute malware. Even if the URL listed on the ad shows a legitimate website, subsequent ad traffic can easily lead to a fake page. Different types of malware are distributed in this manner. I've seen IcedID (Bokbot), Gozi/ISFB, and various information stealers distributed through fake software websites that were provided through Google ad traffic. I submitted malicious files from this example to VirusTotal and found a low rate of detection, with some files not showing as malware at all. Additionally, domains associated with this infection frequently change. That might make it hard to detect.
https://clickercounter.org/
https://defineprogramming.com/
Dec 26th 2022
9 months ago
rthrth
Jan 2nd 2023
8 months ago