Pushdo/Cutwail Spambot - A Little Known BIG Problem

Published: 2009-11-13
Last Updated: 2009-11-13 02:56:14 UTC
by Deborah Hale (Version: 1)
13 comment(s)

Today was another one of those days that all ISP's dread.  I am the Abuse Coordinator for a small Midwestern ISP.  Several days ago we started receiving Spam Abuse reports on the IP address to our Corporate firewall.  Unfortunately,  the IP I discovered is blocklisted on several blocklists.  I began to investigate what could be causing these reports of abuse.  I reviewed the logs in the firewall and discovered that we had a couple of workstations doing some bad things.  Our It techs began to look at the computers (both of which had AV installed) and discovered that we had some pretty significant infections on these computers. Both machines were pulled offline, the data backed up and the machines were formatted and reloaded.  We were pretty confident that we had solved the problem and breathed, an unfortunately premature, sigh of relief. 

Yesterday we again started getting abuse reports so it was back to the drawing board for me.  I started trying to get information on exactly what was being detected and what was causing these abuse reports.  This investigation led me to MultiRBL.org.  We were indeed listed on several blocklists again.   As I began to look at the various blocklists looking for the answers it became apparent that we will dealing with a Trojan/Botnet called Cutwail Spambot aka Pushdo aka Pandex.  The interesting thing is, I hadn't never heard of it. So last night I began to research just what this Cutwail Spambot was.  What I find out just blew me away.

I came across an article from Trend Micro Researchers Alice Decker, David Sanchog, Loucif Kharouni, Max Goncharov, and Robert McArdle.  The article is titled A Study of the Pushdo /Cutwail Botnet, An Indepth Analysis. The article indicates that this particular botnet has been around since January 2007 and is the second largest spam botnet on the planet. This particular spambot is believed to be responsible for approximately 7.7 billion spam emails per day making it responsible for 1 out of every 25 spam emails sent world wide. According to the findings of the research team the development team for Pushdo/Cutwail work very hard and used several techniques to keep their program "under the radar".  In the article they outline these techniques which include things like using multiple variants that react a bit differently, remain memory resident, with very little actually written to disk, and frequent updates and changes to the code to prevent discovery.

This article contains an indepth look at the botnet and gives good insight into how to detect and control the botnet.  This article is well worth reading. Other research that I have done indicates the best program to find the Pushdo/Cutwail Spambot is Microsoft's Windows Malicious Software Removal Tool.

Another article - by Matt McCormack entitled "WHEN THE HAMMER FALLS – EFFECTS OF SUCCESSFUL WIDESPREAD DISINFECTION ON
MALWARE DEVELOPMENT AND DIRECTION" gives additional information about the botnet and gives detailed information and instructions for ridding your network of the botnet.

Our tech's have their work cut out for them.  They are going to have to "touch" all 250 employee computers (249 - mine is clean) plus all of our Windows Servers so that we make sure that we get rid of all of the infected computers.  We are also investigating a change in Anti-virus software.  Unfortunately the one we have been using has fallen into the category of less that reliable so now we are trying to decide what we need to switch too.  Now is as good a time as any, after all we are going to have to "touch" every computer.

I am just amazed that this botnet is the 2nd largest in the world, been around for almost 3 years and I am just now dealing with it.  We still haven't figured out how this botnet got started, we aren't sure where it started at, but we do know we can't wait to rid our network of this mess.

Everyone who manages networks no matter what the size needs to read these articles and know what to look for and how to recognize the presence of the botnet.  I for one vote for irradication of this botnet and a reduction of 7.7 Billion spam emails a day.  Sure would make my spam filter easier to manage.  Wouldn't it be great to somehow eliminate these bad guys.

Check out the articles at:

Matt McCormack Article - download.microsoft.com/download/3/8/d/.../McCormack-VB2008.pdf

Trend Micro Article - us.trendmicro.com/imperia/md/content/us/pdf/.../study_of_pushdo.pdf

I would be interested in hearing about other people's experiences with this Botnet and in finding out if you have any good tips for detecting and "killing" the bot.  So let's hear from all of you botherders out there.

 

Deb Hale Long Lines, LLC

13 comment(s)

Comments

Matt McCormack: http://www.microsoft.com/downloads/details.aspx?FamilyID=44bcc45f-7bad-46a2-b661-69d435754a0b&displaylang=en
FWIW I have seen a workstation infected with this before, about a year ago. I can't remember how anyone initially noticed it, but from that point on I have always logged attempted connections from any workstation to port 25/tcp (or 53/udp) of any external IP; since we run internal SMTP and DNS that should never happen anyway. Surely enough the workstation was spamming away happily.

Anyway, with nobody using the machine it could be easily seen that the machine was communicating with some kind of C&C nodes at 94.103.4.217, 174.36.201.82, 69.147.239.106, 208.43.154.226 and 94.103.4.230 (don't remember the port[s], and these IPs are probably not active any more); randomly connecting to one of these as I blocked each one in the NAT gateway. From that point I decided to monitor those IPs in case any other machines were infected with the same malware and trying to use the same C&C nodes, but fortunately it wasn't the case.

Honestly I'm not sure how I got rid of it. I believe it still operated in Safe Mode and would kill diagnostic tools such as procexp.exe as soon as they tried to load.

I seem to remember, that in procexp I saw data relating to the spam generation in the 'Strings' tab for winlogon.exe; the malware had inserted itself into that service's resident image, or something...

In the end I think it was 'gmer' (http://gmer.net/) that told me which drivers/.sys file it was hiding in. After renaming the file, I used 'notmyfault' from SysInternals to invoke an immediate kernel crash, so that the OS could not perform a clean shutdown (which may have allowed the malware time to drop itself to a new file to load on reboot). I've use that technique a few times to avoid having to re-image, which would have been particularly awkward for that particular workstation, but it's something I should get in the habit of doing because intelligent malware these days may be near-impossible to remove any other way.
another thing to thank fireeye for...

http://blog.fireeye.com/research/2008/11/pushdocutwail-control-servers.html
Deborah:

Not to blame the victim, but...

Your firewall policy permits all hosts on your corporate network to send SMTP to the Internet at large? That is a recipe for disaster, as you've seen.

You should have egress filtering in place, either restricting outbound SMTP for the world at large to the single designated outbound MTA on your local network, or restricting your local network hosts sending SMTP to only your designated MTA outside the firewall.

Or am I misunderstanding the description of your environment?
The environment I spoke of did allow SMTP to the Internet at large, but I decided to stop that after the Cutwail incident.

The problem is that more-intelligent malware is able to use local MTA's too, although you could (and I do) apply spam-filtering to outbound mail to detect and/or block this.

In any case, merely logging outbound SMTP traffic (whether you block it or not) from a workstation should warn of infection by spam-sending malware. On larger networks (or something like an University campus) where people may bring their personal computers, it may not be reasonable to block outgoing SMTP.
Deborah's not a 'victim' here, as she's enough of a security professional to be a handler on duty now and again. I tend to reserve victim for civilians hit by this kind of stuff. Pros, we expect it, right? Part of what they pay us for?

I am very interested that at an ISP the corporate hosts could just send email to whomever via whichever mail server. What do you want to bet the same permissive policy was in place for the customers?
John, Peter and all - you are correct. We should have not allowed this to happen. We have 2 separate areas in our "IT" dept. One for internal customers, one for external customers. Internal customer systems are handled by another individual. Preventations were not in place. Now they are. They learned something today. As for our external customers, oh yeah it is controlled.
John, Peter and all - you are correct. We should have not allowed this to happen. We have 2 separate areas in our "IT" dept. One for internal customers, one for external customers. Internal customer systems are handled by another individual. Preventations were not in place. Now they are. They learned something today. As for our external customers, oh yeah it is controlled.
Hi Deb,

Out of curiosity, how was your machine spared? Or did you clean it up yourself? If your machine was spared, you could consider applying similar safeguards to the other hosts in your network.

CP
I, at least could not follow your links. Please give the full links without the "..."

Diary Archives