IPMI: Hacking servers that are turned "off"

IPMI: Hacking servers that are turned "off"
One of the challenge in managing large server farms remotely is how to deal with crashed / hanging servers once the operating system no longer responds. The classic answer is usually a mix of serial consoles, maybe KVM over IP devices and remote power switches. This equipment isn't just expensive, it also takes up valuable rack space, requires power and lots and lots of extra messy cables. To make things easier, Intel came up with "IPMI". the "Intelligent Platform Management Interface". Typically found in servers, versions of it can also be found in desktops targeting enterprise deployments. IPMI is by no means new, an the attack described here isn't new, but I still find that many system admins are not aware of the potential of modern implementations of IPMI (good or bad). Over the years, there have been a number of different IPMI revisions. How much functionality you get depends on the motherboard vendor and the firmware you are using. But there are a few features that are common to pretty much all IPMI implementations: IPMI is active once the server is connected to power. It does not depend on the server to be actually "switched on". IPMI is implemented as a specific circuit on the motherboard. Sometimes, you may find it on an optional plugin board. But it does not require CPU, RAM or other components It may use an existing network card, and doesn't necessarily need a dedicated network card If your operating system supports IPMI, you can use special software on the server to connect to it and use it for example to read the status of various sensors. Check the "openipmi" or "freeipmi" tools if you don't already have them installed. IPMI is useful locally, but its real power comes to play remotely. IPMI version 1.0 was used over serial ports. Its main feature was to be able to remote power cycle as system. You can probably compare this to a kind of "Wake on LAN" but over serial with the ability to turn power off, not just on. This eliminated the need for remote power controllers. As of version 1.5, it was possible to send IPMI messages over IP. The latest version, 2.0, includes support for blade servers, vLans and a number of additional features commonly found in modern networks. In a current server implementing IPMI, you may find a full blown web server able to control the system remotely, including advanced features like flashing firmware. This pretty much does away with the need for a serial interface. However, you will lose the "out of band" character of a serial connection, that many of us count on for security. There are a couple basic steps you can use to secure IPMI: setup a dedicated management network, and limit IPMI to the network card connected to the management network. review the BIOS configuration option for IPMI. If you can't have a physical management network, at least try to use a VLAN if supported. keep IPMI firmware up to date. It may be included in motherboard firmware updates or delivered as a distinct package. eliminate IPMI access over insecure protocols like HTTP. Use HTTPS with proper certificates, or SSH do not use default passwords try to integrate IPMI authentication with existing authentication systems. Options typically include RADIUS and AD. review hardening options your IPMI implementation provides. You may be able to limit access from IP addresses, or turn off various features you do not need inventory servers with IPMI capability Finally as a bonus, a little video showing one recent IPMI implementation: [1] http://www.intel.com/design/servers/ipmi/index.htm ------ Johannes B. Ullrich, Ph.D. SANS Technology Institute Twitter	Johannes 3904 Posts ISC Handler Jun 7th 2012
Subscribe	Jun 7th 2012 8 years ago
I'm struggling to understand how an IPMI would be internet-accessible by default, given that (in my limited experience) the cards always default to some weird private IP address anyway and would not therefore be reachable?	Anonymous
Quote	Jun 20th 2012 8 years ago
Would IPMI be available from user space?	Chris 6 Posts
Quote	Aug 21st 2013 6 years ago
There are user space utilities (ipmiutil typically) that allow some access to IPMI. A user would have to authenticate to IPMI using the IPMI username/password. IPMI 2.0 allow for key based / encrypted authentication. Locally (not across the network) you typically have to be root to read the IPMI sensor status. (e.g. try "ipmi-sensors" on a unix system. ipmi-sensors -L, which lists available sensors, usually works for normal users. )	Johannes 3904 Posts ISC Handler
Quote	Aug 22nd 2013 6 years ago

One of the challenge in managing large server farms remotely is how to deal with crashed / hanging servers once the operating system no longer responds. The classic answer is usually a mix of serial consoles, maybe KVM over IP devices and remote power switches. This equipment isn't just expensive, it also takes up valuable rack space, requires power and lots and lots of extra messy cables.

To make things easier, Intel came up with "IPMI". the "Intelligent Platform Management Interface". Typically found in servers, versions of it can also be found in desktops targeting enterprise deployments. IPMI is by no means new, an the attack described here isn't new, but I still find that many system admins are not aware of the potential of modern implementations of IPMI (good or bad).

Over the years, there have been a number of different IPMI revisions. How much functionality you get depends on the motherboard vendor and the firmware you are using. But there are a few features that are common to pretty much all IPMI implementations:

IPMI is active once the server is connected to power. It does not depend on the server to be actually "switched on".
IPMI is implemented as a specific circuit on the motherboard. Sometimes, you may find it on an optional plugin board. But it does not require CPU, RAM or other components
It may use an existing network card, and doesn't necessarily need a dedicated network card

If your operating system supports IPMI, you can use special software on the server to connect to it and use it for example to read the status of various sensors. Check the "openipmi" or "freeipmi" tools if you don't already have them installed.

IPMI is useful locally, but its real power comes to play remotely. IPMI version 1.0 was used over serial ports. Its main feature was to be able to remote power cycle as system. You can probably compare this to a kind of "Wake on LAN" but over serial with the ability to turn power off, not just on. This eliminated the need for remote power controllers. As of version 1.5, it was possible to send IPMI messages over IP. The latest version, 2.0, includes support for blade servers, vLans and a number of additional features commonly found in modern networks.

In a current server implementing IPMI, you may find a full blown web server able to control the system remotely, including advanced features like flashing firmware. This pretty much does away with the need for a serial interface. However, you will lose the "out of band" character of a serial connection, that many of us count on for security. There are a couple basic steps you can use to secure IPMI:

setup a dedicated management network, and limit IPMI to the network card connected to the management network.
review the BIOS configuration option for IPMI. If you can't have a physical management network, at least try to use a VLAN if supported.
keep IPMI firmware up to date. It may be included in motherboard firmware updates or delivered as a distinct package.
eliminate IPMI access over insecure protocols like HTTP. Use HTTPS with proper certificates, or SSH
do not use default passwords
try to integrate IPMI authentication with existing authentication systems. Options typically include RADIUS and AD.
review hardening options your IPMI implementation provides. You may be able to limit access from IP addresses, or turn off various features you do not need
inventory servers with IPMI capability

Finally as a bonus, a little video showing one recent IPMI implementation:

[1] http://www.intel.com/design/servers/ipmi/index.htm

------

Johannes B. Ullrich, Ph.D.
SANS Technology Institute
Twitter

Johannes

3904 Posts
ISC Handler

Jun 7th 2012

I'm struggling to understand how an IPMI would be internet-accessible by default, given that (in my limited experience) the cards always default to some weird private IP address anyway and would not therefore be reachable?

Anonymous

Would IPMI be available from user space?

Chris

6 Posts

There are user space utilities (ipmiutil typically) that allow some access to IPMI. A user would have to authenticate to IPMI using the IPMI username/password. IPMI 2.0 allow for key based / encrypted authentication. Locally (not across the network) you typically have to be root to read the IPMI sensor status. (e.g. try "ipmi-sensors" on a unix system. ipmi-sensors -L, which lists available sensors, usually works for normal users. )

Johannes

3904 Posts
ISC Handler