Threat Level: green Handler on Duty: Didier Stevens

SANS ISC: Oops - You Mean That Deleted Server was a Certificate Authority? - SANS Internet Storm Center SANS ISC InfoSec Forums


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!
Oops - You Mean That Deleted Server was a Certificate Authority?

I was recently working at a client, implementing wireless.  As in many Enterprise Wireless projects, we needed an Enterprise Certificate Authority (CA).  Imagine my surprise, that when we went to create an Enterprise Root CA, that one already existed.  And when we went to take a closer look at that Root CA, when we found that the server was retired - dead and gone, I got that sinking feeling and realized we might be on a trip down the project-over-run rabbit hole.

While you can certainly inventory all the certificates issued and active on a Certificate Authority, if the CA is gone there isn't a good way to do that.  So while you can easily delete a Root CA from Active Directory, once you delete it, that CA is no longer in the list of Trusted Roots.  All the Certificates issued by it will be invalid, and in this case nobody really was sure what that CA was put in to do.  So what we needed was an idea of what the impact of deleting that CA might be.

Then I remembered the story I wrote a while back on Microsoft certutil ( https://isc.sans.edu/diary/11962 ).  With a bit of playing, I was able to use certutil and psexec ( from Mark R's excellent  Sysinternals Utilities) to inventory the "Local Computer" certificate store of every machine in the domain.  

Luckily, in this case we only needed to worry about machines in the Active Directory Domain, so this survey got the job done for us.

What we needed to run was a short script like this, on each machine in the domain:

REM ============== getiss.cmd ==============
echo ========================== >> \\utilserver\sharename\certs.txt
hostname >> \\utilserver\sharename\certs.txt
certutil -store my | find "Issuer" >> \\utilserver\sharename\certs.txt

The first 2 lines (the echo and hostname commands) just break up the output, and identify the machine being evaluated in each test.  The last line is where all the action is - we're dumping the local certificate store, only looking at the Local Machine store.  In this case all we're only interested in is which server issued the certificate, so we're looking for the word "Issuer" in the output.  Since we're looking anyway, I'm not going to parse this out further, I'll happily look at *all* the issuers in the domain to see if we've got any other issuer-based certificate problems in our domain.

Now I'll call this little script for every computer in the domain:

psexec \\* -u domainname\adminuser -p adminuserspassword -cf getiss.cmd

Our output looks like:

==========================
KMS
Issuer: CN=CA01-CA, DC=domain, DC=com
==========================
SERVERNAME2
==========================
BACKUPS
==========================
SERVERNAME5
==========================
SERVERNAME6
==========================
SERVERNAME7
Issuer: CN=SERVERNAME7, L=1720207907, OU=SharePoint, O=Microsoft
Root Certificate: Subject matches Issuer
==========================

... and so on.

So what did we find?  The old CA hadn't issued any certificates that were currently in play on anything in the domain.   We also found a number of self-signed certificates (where the Issuer matched the hostname).  So, with this in hand, we can delete that old CA from the Domain and know in our hearts that we're not going to mess up any of the critical services in the organization (Sharepoint or Exchange for instance).  Details on doing this, now that the impact has been assessed, can be found at these and many other links on microsoft.com ( http://support.microsoft.com/kb/555151  , http://support.microsoft.com/kb/889250http://blogs.technet.com/b/pki/archive/2011/10/07/how-to-decommission-a-windows-enterprise-certification-authority-and-how-to-remove-all-related-objects.aspx )

Scripting saves the day again, in about 10 minutes no less !

If you've had a similar experience, or if you've got a simpler or more elegant scripting approach for this type of problem, by all means use our comment form and share.

===============
Rob VandenBrink
Metafore

Rob VandenBrink

497 Posts
ISC Handler
There are other stores besides the local computer certificate store. What about the other stores? For example the user certificate store on a computer:

certutil -store -user my

certutil command reference - http://technet.microsoft.com/library/cc732443.aspx#BKMK_Store
display certs by logical certificate store - http://technet.microsoft.com/en-us/library/cc770851.aspx
Anonymous
Yes, you could certainly dig through each certificate store in this way. In this case we were most concerned about the impact to critical services such as Exchange or SQL, so the computer certificates were the ones we were most interested in.
Rob VandenBrink

497 Posts
ISC Handler
One thing people should keep in mind about psexec and pstools is that they transmit your admin passwords across the network unencrypted, so they could be captured by attackers or malware on the network. You can avoid this by using Windows networking to connect to the remote systems (via the \\server UNC in Windows Explorer or the net use command) before running the pstool.

Also, some people recommend using different local admin passwords on each different computer or groups of computers, as otherwise one compromised password gives the attacker rights to every system.

http://computer-forensics.sans.org/blog/2010/06/01/protecting-admin-passwords-remote-response-forensics/
Rob VandenBrink
8 Posts
What Karl says is true. I really liked Mike Pilkington's follow up article on PSEXEC.

http://computer-forensics.sans.org/blog/2012/12/17/protecting-privileged-domain-accounts-psexec-deep-dive

What I've been doing is creating an elevated CMD window on the local host and then doing all of my PSEXEC-ing from there. Like this:

psexec -u domain\adminname cmd

The resulting CMD window will be running under the admin user. PSEXEC can then be used in that window without passing the PW in plain text. Of course, the same thing can be accomplished by Shift-Right clicking on the Command Prompt icon and running as a different user.
Rob VandenBrink
3 Posts

Sign Up for Free or Log In to start participating in the conversation!