Threat Level: green Handler on Duty: Didier Stevens

SANS ISC: InfoSec Handlers Diary Blog - When Lightning Strikes InfoSec Handlers Diary Blog


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!

When Lightning Strikes

Published: 2010-08-03
Last Updated: 2010-08-04 18:05:09 UTC
by Johannes Ullrich (Version: 1)
11 comment(s)

This weekend, I had a pretty bad lightning strike hit my house. The kind of where you see spark hitting the street in front of the house and your dog jumping in your lap lightning strike. Overall, lightning is a pretty common phenomenon around here. I live in Florida, which appears to be #1 in lightning strikes and casualties in the US [1] . For the 5+ years I live here, the power grid has actually been rather stable during lightning storms, but lately, I had a string of bad luck and would like to share some lessons learned:

So far, I had no damage to equipment completely protected by a UPS/surge protector. I use various types of UPSs, and all performed well so far. Some are rather old and have hardly any battery life left. But they do still work well enough for power spikes/dips as they show up during electrical storms.

The damage I had, in particular in the last storm, affected exclusively network equipment and networking interfaces. I assume that the surge entered the network. I lost two switches and the wired network interfaces in two PCs. Otherwise, the PCs work fine. So far I had not used any network surge protectors, but now started to use the surge protectors provided by the UPS. This appears to work fine, but in some cases, the network now works as "half duplex" and no longer in "duplex" mode. I looked into stand alone network surge protectors for some devices, and it turned out to be a bit hard to find one that supports gigabit ethernet. But they are available. The UPS network surge protection is only supposed to work up to 100 Base-T but synced fine at Gigabit (no duplex).

A thunderstorm a couple months ago, caused some "interesting" damage to my cable modem. I was only able to upload 1MByte in a single connection. This was very weird as it also applied to connections inside VPN tunnels, the cable modem shouldn't really "see" what was happening. But sure enough, swapping the modem fixed the problem. I added a surge protector for the cable line as well. One reason I had not done this before was that I had bad experience with surge protectors and cable modems in the past. But my new cable modem (like many others) provides a status screen and the signal-to-noise number did not suffer significantly after adding the surge protector. The surge protector replaced a simple straight through connector which may have caused a similar loss.

Couple other hints:

- do not plug surge protectors into a UPS. If the UPS runs on batteries it will usually generate a steep sine wave which may destroy surge protectors (in particular tricky to find power strips without surge protector)
- do not plug a UPS into a UPS (same reason as above)
- lightning damage can be subtle. None of my equipment has any visible damage
- proper grounding of all lines entering the house is important (around here, I find that utility companies are pretty good about that)
- once the power is out, turn off the main fuse to the house. But be aware the main fuse can be hard to "flip". Depending on the nature of the outage you may have some surges and unstable power until the damage is repaired (if you want to know when power comes back, just flip all the individual fuses other then one or two that only power lights)

If you consider a backup generator: I looked at many options, but haven't been able to justify one so far. This last outage was 10 hrs long and was by far the longest I have seen. My backup plan is a well charged laptop and a 3G data card to keep me connected. If you consider backup power for a server room, don't forget the AC! For the generators I looked at, the cost to install was almost as much if not more then the cost of the generator. If you do use a portable generator to power individual devices, make sure you do NOT plug the generator into your house wiring before disconnecting the main fuse.

As a quick summary: Surge protectors work. They will probably not save your equipment if the lightning storm rips the electrical wiring out of your walls, but they can help against some pretty nasty strikes. Unplugging your equipment (and WiFi :) ) is better, but not always feasible.

[1] http://www.srh.noaa.gov/mlb/?n=lightning_stats

------
Johannes B. Ullrich, Ph.D.
SANS Technology Institute
Twitter

11 comment(s)
Diary Archives