Threat Level: green Handler on Duty: Xavier Mertens

SANS ISC: Intel Network Card (82574L) Packet of Death - SANS Internet Storm Center SANS ISC InfoSec Forums


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!
Intel Network Card (82574L) Packet of Death

An interesting blog post by Kristian Kielhofer describes how a specific SPI packet can "kill" an Intel Gigabit ethernet card [1]. If a card is exposed to this traffic, the system has to be physically power cycled. A reboot will not recover the system. 

The network card crashed whenever the value 0x32 or 0x33 was found at offset 0x47f. Kristian first noticed this happening for specific SIP packets, but in the end, it turned out that any packet with 0x32 at 0x47f caused the crash. Intel traced the problem to an EEPROM used in this specific card (82574L). There are some links in the comment to the blog suggesting that others have run into this problem before. For example, the commend: "ping -p 32 -s 1110 x.x.x.x" can crash an affected card remotely.

[1] http://blog.krisk.org/2013/02/packets-of-death.html

------
Johannes B. Ullrich, Ph.D.
SANS Technology Institute
Twitter

I will be teaching next: Intrusion Detection In-Depth - SIEM Summit & Training 2019

Johannes

3631 Posts
ISC Handler
Actually, it's just receiving a few particular values, in a particular position in the frame, - so a web page, or even just a modified image could trigger this.

Nasty, but probably worse case is as a DoS attack tool, as a crafted request could trigger the bug.
His case was well, really obscure (a VoIP packet happened to trigger it), but only makes the story more interesting.
DomMcIntyreDeVitto

41 Posts
Some of the newer posts in Kristians link above refer to similar problems with ZTE MMC 3G USB Mobile Broadband dongles and their drivers. I have had similar things occur (BSOD driver failures) lately that require a full reboot and they seemed to have stopped after a Java update. There still seems to be some partial issues where an individual page on a website blows up IE but just closing down the offending page and not IE fixes the problem. Weird to say the least.
Anonymous
Actually the ZTE MMC is my USB not the net card driver failures referred to in the link FYI.
Anonymous
If it is only on a plug-in NIC, one can always change the NIC, but is there any chance the same bug exists on motherboard mounted NICs also? That would require replacing the entire motherboard -- unless it is on-site reflashable and Intel has a patch.
Moriah

133 Posts
It surely can't be that easy? Or else a stream of encrypted (pseudorandom) data would trigger the bug after approx. 128 packets.

One of my server rentals has this model of NIC onboard. I sure hope they're not vulnerable, or else my provider has a whole datacentre full of these...
Steven C.

171 Posts
- http://www.securitytracker.com/id/1028089
"... A power cycle is required to return the system to normal operation.
The original advisory is available at:
http://blog.krisk.org/2013/02/packets-of-death.html
Impact: A remote user can cause the target controller to crash.
Solution: The vendor has issued a fix, available via customer support...
[Rots o' ruck finding that number.]
.
Jack

160 Posts
FYI:

I used the two pcaps ( pod-http-post.pcap & pod-icmp-ping.pcap ) from http://www.kriskinc.com/intel-pod using the tcpdump-edit instructions provided. I also added "-l 30" to retry.

I have an Intel 82574L Gigabit CT adapter on an ASUS X79 board running CentOS 6.3, fully patched. The box says "Version E39199-008" "Date: 09/08/2012". I have been unable to duplicate the crash.

I "ifconfig up"ed the adapter (without an IP). "tcpdump -p -i eth3" shows the machine receiving the frames, but it doesn't stop working.
MarkJx

5 Posts
Intel technical support number: 916-377-7000
JeffSoh

31 Posts
We uploaded the offending packet to our online packet viewer, CloudShark, with some notes on what to look for and links to Kristian's articles on the topic. Check it out here:

http://appliance.cloudshark.org/news/cloudshark-in-the-wild/intel-packet-of-death-capture/
JeffSoh
1 Posts
I wonder if there's any chance the scope of this problem is bigger than 82574L

The symptoms sound very similar to an issue I have been having with an Intel 82567LM card over the passed 2 years. (In a Dell Optiplex 960)

It just randomly just quits talking on the network....and to fix it you can either reboot, or unplug the NIC cable for 10 seconds and then plug it back in. The problem seems to semi-randomly happen after the sleeping computer wakes up

While it is in this strange state, sometimes I can PING and sometimes I can't. If I can't ping, then if I IPCONFIG...the static IP Address no longer shows it is registered. IT shows 0.0.0.0

Very Strange!

Motherboard has been changed out 3-4 times (by Dell....who sends me the same motherboard and embedded NIC I'm assuming)
Network cable replaced
Cable run to the closet tested
Different network port used
Drivers/BIOS/chipset drivers updated

I also seem to be having sporadic problems with 82579LM cards that are in systems that were brought online last year. After the system wakes up for its nightly backup, the backup server can not contact the system. Doesn't happen all the time tho.... pretty random and sporadic....
K-Dee

63 Posts
Thanks for updating the article; but if there are 2 values that could trigger this and one that would prevent it, I think it would be quickly triggered on 2 out of 3 hosts. I imagine there's some other requirement for the ethernet frame, and that might limit exposure to being link-local.

Anyway I'm concerned anything like this was even possible. Why is software hidden in firmware blobs inspecting payload data? Remotely-triggerable bugs of this sort could allow DoS of whole data centres, customer premises equipment (modem/routers), or ISPs' core infrastructure.
Steven C.

171 Posts
Re: 'innoculation', Kris K's blog later mentions that most values other than 0x30,31,32 will do that. The chance of hitting the error becomes more like 1 in 128 and probably depends what type of traffic is sent/received during startup, or incoming broadcasts on your network.
Steven C.

171 Posts
I guess I'm a bit more cynical than others, looking at a weak link that was produced back in 2012 and Intel's pass the buck response. Personally I've seen bad code in their tethered and non-tethered products through HP, Dell.

Intel "worked with the author as well as the original motherboard manufacturer to investigate and determine root cause"? Intel caused the issue, not the MB manufactures with a flawed EEPROM image that could have been fixed.

We had 5 servers that ran dual NICS with this chipset.. Bravo!

A lot of us have been around this industry for years, Sadly what is becoming a trend as new products come out, there is some security flaw to them. If we did work like this, we would not be working.

scsmith59

6 Posts

Sign Up for Free or Log In to start participating in the conversation!