My next class:

Intel Network Card (82574L) Packet of Death

Published: 2013-02-06. Last Updated: 2013-02-08 21:07:10 UTC
by Johannes Ullrich (Version: 3)
13 comment(s)

[Update]  Intel released a statement about this issue. and Kristian updated his blog

An interesting blog post by Kristian Kielhofer describes how a specific SPI packet can "kill" an Intel Gigabit ethernet card [1]. If a card is exposed to this traffic, the system has to be physically power cycled. A reboot will not recover the system. 

The network card crashed whenever the value 0x32 or 0x33 was found at offset 0x47f. Kristian first noticed this happening for specific SIP packets, but in the end, it turned out that any packet with 0x32 at 0x47f caused the crash. Intel traced the problem to an EEPROM used in this specific card (82574L). There are some links in the comment to the blog suggesting that others have run into this problem before. For example, the commend: "ping -p 32 -s 1110 x.x.x.x" can crash an affected card remotely.

[Update] A few asked why this doesn't happen just randomly every 128th packet: Once the card receives the value "0x34" in this position, it appears to be no longer vulnerable. There are also a number of earlier bug reports about this card that sound very similar, and appear to be related to ASPM, a PCI power safe feature. Kristian claims he eliminated this issue. if you try to reproduce this issue, power up the system and then issue the "ping" command shown above quickly after reboot in order to avoid the "inoculation" wiht 0x34. We would like to hear any reports of being able to reproduce (or not) this issue. 

There are also some reports about similar issues in certain 3G USB modems.

 

[1] http://blog.krisk.org/2013/02/packets-of-death.html

------
Johannes B. Ullrich, Ph.D.
SANS Technology Institute
Twitter

Keywords: ethernet card intel
13 comment(s)
My next class:

Comments

Actually, it's just receiving a few particular values, in a particular position in the frame, - so a web page, or even just a modified image could trigger this.

Nasty, but probably worse case is as a DoS attack tool, as a crafted request could trigger the bug.
His case was well, really obscure (a VoIP packet happened to trigger it), but only makes the story more interesting.

Dom De Vitto

Feb 6th 2013
1 decade ago

Some of the newer posts in Kristians link above refer to similar problems with ZTE MMC 3G USB Mobile Broadband dongles and their drivers. I have had similar things occur (BSOD driver failures) lately that require a full reboot and they seemed to have stopped after a Java update. There still seems to be some partial issues where an individual page on a website blows up IE but just closing down the offending page and not IE fixes the problem. Weird to say the least.

Laurie

Feb 7th 2013
1 decade ago

Actually the ZTE MMC is my USB not the net card driver failures referred to in the link FYI.

Laurie

Feb 7th 2013
1 decade ago

If it is only on a plug-in NIC, one can always change the NIC, but is there any chance the same bug exists on motherboard mounted NICs also? That would require replacing the entire motherboard -- unless it is on-site reflashable and Intel has a patch.

Moriah

Feb 7th 2013
1 decade ago

It surely can't be that easy? Or else a stream of encrypted (pseudorandom) data would trigger the bug after approx. 128 packets.

One of my server rentals has this model of NIC onboard. I sure hope they're not vulnerable, or else my provider has a whole datacentre full of these...

Steven Chamberlain

Feb 7th 2013
1 decade ago

- http://www.securitytracker.com/id/1028089
"... A power cycle is required to return the system to normal operation.
The original advisory is available at:
http://blog.krisk.org/2013/02/packets-of-death.html
Impact: A remote user can cause the target controller to crash.
Solution: The vendor has issued a fix, available via customer support...
[Rots o' ruck finding that number.]
.

PC.Tech

Feb 7th 2013
1 decade ago

FYI:

I used the two pcaps ( pod-http-post.pcap & pod-icmp-ping.pcap ) from http://www.kriskinc.com/intel-pod using the tcpdump-edit instructions provided. I also added "-l 30" to retry.

I have an Intel 82574L Gigabit CT adapter on an ASUS X79 board running CentOS 6.3, fully patched. The box says "Version E39199-008" "Date: 09/08/2012". I have been unable to duplicate the crash.

I "ifconfig up"ed the adapter (without an IP). "tcpdump -p -i eth3" shows the machine receiving the frames, but it doesn't stop working.

Mark Jx

Feb 7th 2013
1 decade ago

Intel technical support number: 916-377-7000

JeffSoh

Feb 7th 2013
1 decade ago

We uploaded the offending packet to our online packet viewer, CloudShark, with some notes on what to look for and links to Kristian's articles on the topic. Check it out here:

http://appliance.cloudshark.org/news/cloudshark-in-the-wild/intel-packet-of-death-capture/

CloudShark

Feb 7th 2013
1 decade ago

I wonder if there's any chance the scope of this problem is bigger than 82574L

The symptoms sound very similar to an issue I have been having with an Intel 82567LM card over the passed 2 years. (In a Dell Optiplex 960)

It just randomly just quits talking on the network....and to fix it you can either reboot, or unplug the NIC cable for 10 seconds and then plug it back in. The problem seems to semi-randomly happen after the sleeping computer wakes up

While it is in this strange state, sometimes I can PING and sometimes I can't. If I can't ping, then if I IPCONFIG...the static IP Address no longer shows it is registered. IT shows 0.0.0.0

Very Strange!

Motherboard has been changed out 3-4 times (by Dell....who sends me the same motherboard and embedded NIC I'm assuming)
Network cable replaced
Cable run to the closet tested
Different network port used
Drivers/BIOS/chipset drivers updated

I also seem to be having sporadic problems with 82579LM cards that are in systems that were brought online last year. After the system wakes up for its nightly backup, the backup server can not contact the system. Doesn't happen all the time tho.... pretty random and sporadic....

K-Dee

Feb 8th 2013
1 decade ago

Login here to join the discussion.


Top of page
Diary Archives