Published: 2011-08-31
Last Updated: 2011-09-01 05:22:22 UTC
by Johannes Ullrich (Version: 1)
4 comment(s) announced that it was compromised sometime earlier this month [1]. The compromise was discovered on Aug. 28th. At this point, the assumption is that the attacker obtained valid user credentials, and then escalated privileged to become root. The exact nature of the privilege escalation is not known so far.

The attacker apparently managed to modify the OpenSSH client and server on the system, logging user interactions with the server.

It is very unlikely that kernel source code got altered. The kernel source is verified via SHA-1 cryptographic checksums according to the note on No changes were detected.These hashes exist on other machines as well so if an attacker modifies the hash on the server, the change would still be detected.

[an earlier version of this diary stated that the OpenSSH source was modified. This was a misinterpretation of the advisory. Thx Maarten for pointing this out]



Johannes B. Ullrich, Ph.D.
SANS Technology Institute

Keywords: kernelorg linux
4 comment(s)


Hmm...after re-reading the post on a few times...

I think you might have misintepreted their post.

They stated that all source code was fully intact. When they talked about OpenSSH, I think they were referring to the particular installation of OpenSSH on their servers. As a result, you'd only have pulled a trojaned version of OpenSSH if you had directly copied their server's binaries and scripts to your system or a distro repository.

Since distros are compiled from source (and not pulled as binaries from these server), there should be no concern, correct?
ouch... i installed cygwin a few days ago from their mirrors (

i think it was not affected because the cygwin installer does checksum verifications (or is supposed to do that) before trusting the packages downloaded and it did not complain about any mismatch.
---------- Forwarded message ----------
From: J.H. <>
Date: 2011/8/29
Subject: [ users] [KORG] Master back-end break-in

Hash: SHA1

Afternoon Everyone,

As you can guess from the subject line, I've not had what many would
consider a "good" day. Earlier today discovered a trojan existing on
HPA's personal colo machine, as well as hera. Upon some investigation
there are a couple of boxes, specifically hera and odin1,
with potential pre-cursors on demeter2, zeus1 and zeus2, that have been
hit by this.

As it stands right now, HPA is working on cleaning his box, and
I'm working on hera (odin1 and zeus1 are out of rotation still for other
reasons), mainly so that if one of us finds something of interest, we
can deal with it and compare notes on the other box.

Points of interest:

- - Break-in seems to have initially occurred no later than August 12th

- - Files belonging to ssh (openssh, openssh-server and openssh-clients)
were modified and running live. These have been uninstalled and
removed, all processes were killed and known good copies were
reinstalled. That said all users may wish to consider taking this
opportunity to change their passwords and update ssh keys (particularly
if you had an ssh private key on hera). This seems to have occurred on
or around August 19th.

- - A trojan startup file was added to rc3.d

- - User interactions were logged, as well as some exploit code. We have
retained this for now.

- - Trojan initially discovered due to the Xnest /dev/mem error message
w/o Xnest installed; have been seen on other systems. It is unclear if
systems that exhibit this message are susceptible, compromised or not.
If you see this, and you don't have Xnest installed, please investigate.

- - It *appears* that 3.1-rc2 might have blocked the exploit injector, we
don't know if this is intentional or a side affect of another bugfix or

- - System is being verified from backups, signatures, etc. As of right
now things look correct, however we may take the system down soon to do
a full reinstall and for more invasive checking.

- - As a precaution a number of packages have been removed from the
system, if something was removed that you were using please let us know
so we can put it back.

- - At this time we do not know the vector that was used to get into the
systems, but the attackers had gained root access level privileges.

That's what we know right now, some of the recent instabilities may have
been caused by these intrusions, and we are looking into everything.

If you are on the box, keep an eye out, and if you see something please
let us know immediately.

Beyond that, verify your git trees and make sure things are correct.

- - John 'Warthog9' Hawley
Chief Administrator
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Fedora -

Perhaps everyone needs to start running TripWire or AIDE every day.

Diary Archives