Another Network Forensic Tool for the Toolbox

Another Network Forensic Tool for the Toolbox - Dshell
This is a guest diary written by Mr. William Glodek – Chief, Network Security Branch, U.S. Army Research Laboratory As a network analysis practitioner, I analyze multiple gigabytes of pcap data across multiple files on a daily basis. I have encountered many challenges where the standard tools (tcpdump, tcpflow, Wireshark/tshark) were either not flexible enough or couldn’t be prototyped quickly enough to do specialized analyzes in a timely manner. Either the analysis couldn’t be done without recompiling the tool itself, or the plugin system was difficult to work with via command line tools. Dshell, a Python-based network forensic analysis framework developed by the U.S. Army Research Laboratory, can help make that job a little easier [1]. The framework handles stream reassembly of both IPv4 and IPv6 network traffic and also includes geolocation and IP-to-ASN mapping data for each connection. The framework also enables development of network analysis plug-ins that are designed to aid in the understanding of network traffic and present results to the user in a concise, useful manner by allowing users to parse and present data of interest from multiple levels of the network stack. Since Dshell is written entirely in Python, the entire code base can be customized to particular problems quickly and easily; from tweaking an existing decoder to extract slightly different information from existing protocols, to writing a new parser for a completely novel protocol. Here are two scenarios where Dshell has decreased the time required to identify and respond to network forensic challenges. Malware authors will frequently embed a domain name in a piece of malware for improved command and control or resiliency to security countermeasures such as IP blocking. When the attackers have completed their objective for the day, they minimize the network activity of the malware by updating the DNS record for the hostile domain to point to a non-Internet routable IP address (ex. 127.0.0.1). When faced with hundreds or thousands of DNS requests/responses per hour, how can I find only the domains that resolve to a non-routable IP address? Dshell> decode –d reservedips *.pcap The “reservedips” module will find all of the DNS request/response pairs for domains that resolve to a non-routable IP address, and display them on a single line. By having each result displayed on a single line, I can utilize other command line utilities like awk or grep to further filter the results. Dshell can also present the output in CSV format, which may be imported into many Security Event and Incident Management (SEIM) tools or other analytic platforms. A drive-by-download attack is successful and a malicious executable is downloaded [2]. I need to find the network flow of the download of the malicious executable and extract the executable from the network traffic. Using the “web” module, I can inspect all the web traffic contained in the sample file. In the example below, a request for ‘xzz1.exe’ with a successful server response is likely the malicious file. I can then extract the executable from the network traffic by using the “rip-http” module. The “rip-http” module will reassemble the IP/TCP/HTTP stream, identify the filename being requested, strip the HTTP headers, and write the data to disk with the appropriate filename. There are additional modules within the Dshell framework to solve other challenges faced with network forensics. The ability to rapidly develop and share analytical modules is a core strength of Dshell. If you are interested in using or contributing to Dshell, please visit the project at https://github.com/USArmyResearchLab/Dshell. [1] Dshell – https://github.com/USArmyResearchLab/Dshell [2] http://malware-traffic-analysis.net/2015/01/03/index.html I will be teaching next: Defending Web Applications Security Essentials - SANS San Francisco Spring 2020	Johannes 3735 Posts ISC Handler
Subscribe	Feb 3rd 2015 4 years ago
Great tool! Been using it for about a month now; nice addition. thanks for making it available.	JeffSoh 31 Posts
Quote	Feb 3rd 2015 4 years ago
Pardon my ignorance but how is this different from something like Bro? Is it just another option or does it have any particularly unique features?	JeffSoh 1 Posts
Quote	Feb 3rd 2015 4 years ago
I have used this tool in the last weeks many times. It was and is very helpful to analyse who is trying to access my network to compromise it. What I am missing is a better documentation to use the whole functionality it provides and a description how I can enhance the functionality (means a tutorial).	JeffSoh 2 Posts
Quote	Mar 14th 2015 4 years ago
In my understanding Bro is more a intrusion detection system. Dshell is a network analyser. It has no features to disallow any traffic, e.g by rules or similar like that.	JeffSoh 2 Posts
Quote	Mar 14th 2015 4 years ago

This is a guest diary written by Mr. William Glodek – Chief, Network Security Branch, U.S. Army Research Laboratory

As a network analysis practitioner, I analyze multiple gigabytes of pcap data across multiple files on a daily basis. I have encountered many challenges where the standard tools (tcpdump, tcpflow, Wireshark/tshark) were either not flexible enough or couldn’t be prototyped quickly enough to do specialized analyzes in a timely manner. Either the analysis couldn’t be done without recompiling the tool itself, or the plugin system was difficult to work with via command line tools.

Dshell, a Python-based network forensic analysis framework developed by the U.S. Army Research Laboratory, can help make that job a little easier [1]. The framework handles stream reassembly of both IPv4 and IPv6 network traffic and also includes geolocation and IP-to-ASN mapping data for each connection. The framework also enables development of network analysis plug-ins that are designed to aid in the understanding of network traffic and present results to the user in a concise, useful manner by allowing users to parse and present data of interest from multiple levels of the network stack. Since Dshell is written entirely in Python, the entire code base can be customized to particular problems quickly and easily; from tweaking an existing decoder to extract slightly different information from existing protocols, to writing a new parser for a completely novel protocol. Here are two scenarios where Dshell has decreased the time required to identify and respond to network forensic challenges.

Malware authors will frequently embed a domain name in a piece of malware for improved command and control or resiliency to security countermeasures such as IP blocking. When the attackers have completed their objective for the day, they minimize the network activity of the malware by updating the DNS record for the hostile domain to point to a non-Internet routable IP address (ex. 127.0.0.1). When faced with hundreds or thousands of DNS requests/responses per hour, how can I find only the domains that resolve to a non-routable IP address?

Dshell> decode –d reservedips *.pcap

The “reservedips” module will find all of the DNS request/response pairs for domains that resolve to a non-routable IP address, and display them on a single line. By having each result displayed on a single line, I can utilize other command line utilities like awk or grep to further filter the results. Dshell can also present the output in CSV format, which may be imported into many Security Event and Incident Management (SEIM) tools or other analytic platforms.

A drive-by-download attack is successful and a malicious executable is downloaded [2]. I need to find the network flow of the download of the malicious executable and extract the executable from the network traffic.
Using the “web” module, I can inspect all the web traffic contained in the sample file. In the example below, a request for ‘xzz1.exe’ with a successful server response is likely the malicious file.

I can then extract the executable from the network traffic by using the “rip-http” module. The “rip-http” module will reassemble the IP/TCP/HTTP stream, identify the filename being requested, strip the HTTP headers, and write the data to disk with the appropriate filename.

There are additional modules within the Dshell framework to solve other challenges faced with network forensics. The ability to rapidly develop and share analytical modules is a core strength of Dshell. If you are interested in using or contributing to Dshell, please visit the project at https://github.com/USArmyResearchLab/Dshell.

[1] Dshell – https://github.com/USArmyResearchLab/Dshell
[2] http://malware-traffic-analysis.net/2015/01/03/index.html

I will be teaching next: Defending Web Applications Security Essentials - SANS San Francisco Spring 2020

Johannes

3735 Posts
ISC Handler

Great tool! Been using it for about a month now; nice addition. thanks for making it available.

JeffSoh

31 Posts

Pardon my ignorance but how is this different from something like Bro? Is it just another option or does it have any particularly unique features?

JeffSoh
1 Posts

I have used this tool in the last weeks many times. It was and is very helpful to analyse who is trying to access my network to compromise it. What I am missing is a better documentation to use the whole functionality it provides and a description how I can enhance the functionality (means a tutorial).

JeffSoh
2 Posts

In my understanding Bro is more a intrusion detection system. Dshell is a network analyser. It has no features to disallow any traffic, e.g by rules or similar like that.

JeffSoh
2 Posts