Heartbleed CRL Activity Spike Found
Update: CloudFlare posted in their blog twice today claiming responsibility for the majority of this spike. Quoting: "If you assume that the global average price for bandwidth is around $10/Mbps, just supporting the traffic to deliver the CRL would have added $400,000USD to Globalsign's monthly bandwidth bill."
Update: We've also seen articles from ZDNet and WIRED today in response to the below insights, with further analysis therein.
It looks like, as I had suspected, the CRL activity numbers we have been seeing did not reflect the real volume caused by the OpenSSL Heartbleed bug.
This evening I noticed a massive spike in the amount of revocations being reported by this CRL: http://crl.globalsign.com/gs/gsorganizationvalg2.crl
The spike is so large that we initially thought it was a mistake, but we have since confirmed that it's real! We're talking about over 50,000 unique revocations from a single CRL:
This is by an order of magnitude the largest spike in revocation activity seen in years, according to our current data.
I have set up a new page for everyone to monitor the activity as well as see how we are obtaining this data. The page can be found at https://isc.sans.edu/crls.html.
How will you use this page in your projects or general analysis? We'd love to hear some ideas.
If you know of other CRLs that we can add, please let us know in the comments! Additionally, if you would like to see an API call added so that you can automatically query us for this information, please let us know so that we are aware of the demand.
On a side note, we can see a clear upward trend in revocations over the past 3 or 4 years:
What do you attribute this consistent growth in revocations to? What do you think caused the previous spikes?
--
Alex Stanford - GIAC GWEB,
Research Operations Manager,
SANS Internet Storm Center
/in/alexstanford | @alexstanford
Comments
My guess though is that it will take a while for most people to go through the motions of revoking and reissuing certificates. I can see a whole bunch of reasons for that, such as:
- They can't remember how to request and renew certificates
- They're concerned about cost (and haven't looked into if their CA will issue free certs as a one-off or not)
- They're still building up their list of which systems are affected; OR
- They've decided to wear the risk and not renew their certificates
Anonymous
Apr 17th 2014
1 decade ago
Anonymous
Apr 17th 2014
1 decade ago
My guess though is that it will take a while for most people to go through the motions of revoking and reissuing certificates. I can see a whole bunch of reasons for that, such as:
- They can't remember how to request and renew certificates
- They're concerned about cost (and haven't looked into if their CA will issue free certs as a one-off or not)
- They're still building up their list of which systems are affected; OR
- They've decided to wear the risk and not renew their certificates[/quote]
While I think each bullet you posted does refer to a large portion of people, right now we have no reason to believe that this isn't accurate data. The best guesses we have seen are in the range of 800,000 Heartbleed vulnerable sites. It's possible that the bullets you point to are still the majority and that this spike is only the minority.
Anonymous
Apr 17th 2014
1 decade ago
In a year's time the lessons learned are going to make for great material in GSEC. That's about the only silver lining I can think of.
Anonymous
Apr 17th 2014
1 decade ago
Anonymous
Apr 17th 2014
1 decade ago
There is also likely a delay in having a certificate added after it is revoked to give the organization enough time to swap certificates.
As far as Globalsign goes, I think we see a big hit from them because they are working with Cloudflare. If you "proxy" an SSL site via Cloudflare, Cloudflare is going to request a certificate for it. So as Cloudflare became aware of Heartbleed (and they were ahead of the game on it), the likely issued all new certificates which in itself could explain the 100k or such revocations that came through yesterday.
Anonymous
Apr 17th 2014
1 decade ago
Perhaps the early 2013 spike is some large organization rolling their certificates in response to Heartbleed?
Anonymous
Apr 18th 2014
1 decade ago
The 2013 CRL tsunami might be attributed to the news that Verisign had been hacked:
http://www.pcworld.com/article/249242/verisign_hacked_what_we_dont_know_might_hurt_us.html
Just my surmise on both counts.
Anonymous
Apr 18th 2014
1 decade ago
Thanks for all of the comments everyone, it's great to see more new names in the forums!
Anonymous
Apr 18th 2014
1 decade ago