Tomcat IR with XOR.DDoS
Apache Tomcat is a java based web service that is used for different applications. While you may have it running in your environment, you may not be familiar with its workings to provide adequate incident response when the time come. This article will walk through an incident where Tomcat is used and what critical artifacts you should collect.
This articles assumes that you have already collected MACtimes (hxxps://digital-forensics.sans.org/blog/2011/12/07/digital-forensic-sifting-super-timeline-analysis-and-creation) and other volatile data.
Locating Install Directory
Tomcat can be installed in several different locations, but the most common are for Redhat: /usr/share/apache-tomcat-(version) and for Ubuntu: /usr/share/tomcat or /var/lib/tomcat. To be sure you have the right locations track down the Tomcat process and analyze it.
#ps -auxwww |fgrep -i tomcat
0 S root 31847 1 0 80 0 - 1124641 futex_ 2015 ? 02:36:33 /usr/bin/java -classpath /usr/share/apache-tomcat-7.0.65/bin/bootstrap.jar …..
Here you can see that it is running from /usr/share/apache-tomcat-7.0.65. Depending on the install this will likely contain all the artifacts for tomcat that we need but we should still verify. If you are not getting an entire copy of the system, make sure that you get a backup of this directory along with the other critical ones.
Tomcat Logs
On the system I dealt with, the logs were in a subfolder called logs (e.g. /TOMCAT_HOME/logs). Check to make sure you have stuff in it, if not it is likely using syslog or the attacker cleared it out.
Catalina.out file is output from the tomcat console into this file. It is a multi-line log file, but has time and date stamps. This file shows if modules are added and other output that is useful for IR. Access_log is in common Apache format, but it is a simple format that does not have the user agent information. The manager.log and host-manager.log really did not have any useful information in my case.
For more information about Tomcat loging visit hxxps://tomcat.apache.org/tomcat-7.0-doc/logging.html
Web Applications Locations
Web applications are located on TOMCAT_HOME/webapps/. To deploy a web app, you must use the Tomcat web manager which you will see GETS and POSTS to /manager/ folder in the access logs.
For more info on Tomcat app deployment visit hxxps://tomcat.apache.org/tomcat-4.1-doc/appdev/deployment.html
Config File Locations
The Tomcat configurations are located in the TOMCAT_HOME/config directory. Server.xml is the main server config file. Tomcat-users.xml is a list of users and passwords for the system. The default admin password is:admin:password.
The Incident
Now that we know where to look, let’s go over the incident. A system was discovered to be compromised so I started our IR process. When looking at the processes running, a process was quickly changing its name and running as root. When looking at the list of open files for the process, I got a hint that tomcat might have something to do with the compromise.
#ps -auxwww
10346 root 20 0 21852 952 212 S 25.9 0.0 7103:14 qymasclksx
#lsof -p 10346
qymasclks 10346 root DIR 253,6 4096 275525 /usr/share/apache-tomcat-7.0.65/bin
qymasclks 10346 root rtd DIR 253,0 4096 2 /
qymasclks 10346 root txt REG 253,6 619090 3154 /usr/bin/qymasclksx
Lets see if any files have changed in the Tomcat directory recently to get an idea of possible time of compromise. The file in /usr/bin/qymasclksx is being deleted and recreated every few seconds, so that will not be helpful for initial compromise time.
Tue Dec 01 2015 05:58:38,493137,mac.,-rw-r--r--,0,0,0,"/usr/share/apache-tomcat-7.0.65/webapps/eei.war"
Tue Dec 01 2015 05:58:38,69334,.ac.,-rw-r--r--,0,0,0,"/usr/share/apache-tomcat-7.0.65/webapps/eei/a.jsp"
There is a new file eei.war that has been created. Lets take a look at the log files and see what we can get from that time frame.
#fgrep “Dec 01, 2015” Catalina.out
Dec 01, 2015 5:58:38 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive /usr/share/apache-tomcat-7.0.65/webapps/eei.war
Dec 01, 2015 5:58:38 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deployment of web application archive /usr/share/apache-tomcat-7.0.65/webapps/eei.war has finished in 118 ms
You can see that a new application has been deployed, which means the attacker had access to the Tomcat admin. Let look at the access_logs to see if we can get more detail.
#fgrep “01/Dec/2015” access_logs
122.236.51.194 - - [01/Dec/2015:05:58:08 -0500] "GET /manager/html HTTP/1.1" 401 2474
122.236.51.194 - admin [01/Dec/2015:05:58:09 -0500] "GET /manager/html HTTP/1.1" 200 19270
122.236.51.194 - admin [01/Dec/2015:05:58:39 -0500] "POST /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=4C0343589816E985E2010C618944EF5A HTTP/1.1" 200 20940
122.236.51.194 - - [01/Dec/2015:05:58:43 -0500] "GET /eei/ HTTP/1.1" 200 3319
122.236.51.194 - - [01/Dec/2015:05:58:45 -0500] "POST /eei/ HTTP/1.1" 200 6383
122.236.51.194 - - [01/Dec/2015:05:58:49 -0500] "GET /eei/?action=command HTTP/1.1" 200 2677
122.236.51.194 - - [01/Dec/2015:05:58:55 -0500] "POST /eei/?action=command HTTP/1.1" 200 2736
122.236.51.194 - - [01/Dec/2015:05:58:58 -0500] "POST /eei/?action=command HTTP/1.1" 200 2725
Here you can see that the attacker accessed the Tomcat management as the admin user and uploaded a file. Then the attacker access another page that accepts a command via a POST. Let’s see how the attacker was able to gain access as the admin user to the manager site. By viewing the tomcat-users.xml file, we can see that a very common username/password is being used.
/usr/share/apache-tomcat-7.0.65/conf/tomcat-users.xml
-->
The package that was installed was “jsp File browser 1.1a”. This allowed the attacker to install his backdoor/DDOS tool called Xorddos. Mandint did a write-up about this and mostly what I found was similar to their findings. (hxxps://www.fireeye.com/blog/threat-research/2015/02/anatomy_of_a_brutef.html)
Other IOC(s)
MD5 filehash:
968c4e06ff32d97b1f2b3f2ce3bcbb7e, gfty
a568167a5950ae55f6f442d959de4476, libkill.so
cac58ebacb036f706d58ec9f47ee64cc, eei.war
filename:
c.rar
libkill.so
cmd.n
eei.war
cron.sh
kill.sh
filepath:
/usr/share/apache-tomcat/webapps/eei
/lib
/etc/cron.hourly/
/lib/udev
IPS:
222.186.21.166 (Download site w/ wget)
122.236.51.194 (Attacker IP)
23.234.60.143 (C.rar download every 30 min)
--
Tom Webb
Comments
Going back over 15 years (to Tomcat 4.0.x - I didn't check back further) Tomcat has NEVER shipped with a user that has access to the manager application. There is not, and has never been, a default user name and password for any administrative web application that ships with Tomcat.
Tomcat 4.0.x through 5.5.x shipped with a small number of test users enabled but these were only used by the examples web application and could not do anything harmful.
As of Tomcat 6.0.x (almost a decade ago) even the example users were commented out.
It has always been necessary to manually create a new user or add the appropriate role to an existing user in order to access the manager application.
The article should be corrected to state that "A user with a weak password had been configured for the Manager application".
Note that in the unlikely case that one of the Linux distributions is distributing Tomcat with a pre-configured default user and password for the Manager application then a security vulnerability would need to be raised against the distribution.
Anonymous
Feb 13th 2016
8 years ago
Anonymous
Feb 14th 2016
8 years ago