Is "Reputation Backscatter" a Thing?

Published: 2013-09-03
Last Updated: 2013-09-03 16:41:28 UTC
by Rob VandenBrink (Version: 1)
3 comment(s)

I recently migrated a client from a 10mbps internet uplink to a new 100mbps uplink with a wireless 10mbps backup.  As part of this, they of course got new IP addresses.

Like the thorough, some would say compulsive person I am, before we migrated I did all the right things:

  • Tested both uplinks to make sure they were working
  • Be sure that I had access to ISP support for both uplinks
  • shortened the DNS TTL to ensure that when we migrated our DNS changes would propogate quickly
  • Checked the IP addresses for SMTP Blocklisting (more on this later).

As expected, the migration went smoothly.  Until the next morning.   My client called me bright and early, with the news  "Our users can't send email to company XYZ".  After some wrangling and some time, I got the NDR (Non Delivery Report).  By then, we had identifed 3 other organizations that would not receive our emails.

The key line in the NDR was:
#< #5.7.1 smtp; 550 5.7.1 Service unavailable; Client host [x.x.x.x] blocked using Blocklist 1; To request removal from this list please forward this message to delist@messaging.microsoft.com> #SMTP#

How could this be?  These IP addresses hadn't been used in at least 6 months!

After a bit of digging (Google really does know all), we found that this is the blocklist service employed by Microsoft Office 365.  This service is unique amongst email blocklist services in that there is no way to check your status online, so me checking in advance with MXTOOLS, Solarwinds EE or any of the other usual tools had not done me a bit of good.

Anyway, we emailed the indicated address with our problem, and asked to be removed from the list.
It soon became apparant that this blocklist service was unique in another important way.  The users of the system of course thought that this email problem was our problem.   From our perspective, the solution to the problem had to be implemented by their mail provider.  The roadblock we had was that, as far as they blocklist was concerned, *they* were the Microosft customer, not us.  So as far as the blocklist admins were concerned, we were nobody.

So, like every other blocklist service under the sun, 6 hours went by, then 12, then 18, and still no word.  We ended up having to open a paid support ticket to get ourselves off a list we never should have been on in the first place.

What did I learn?  That cloud services aren't all sunshine and lolipops?  Umm, no, I already knew that.  That Murphy (as in Murphy's Law) is great at exploiting new features and services?  I thought I knew that too, I just though I had it covered (that'll teach me !! )

The important lesson I learned (aside from the "Murphy lesson") was to add one more check in any migrations that affect email - send a test note to anyone of Office 365.

Have you had similar experiences with email migrations?  Or other gotcha's you though you had 100% covered, but not so much?  Use our comment form to let us know what problems you ran into, and how you resolved them.

===============
Rob VandenBrink
Metafore

3 comment(s)
ISC StormCast for Tuesday, September 3rd 2013 http://isc.sans.edu/podcastdetail.html?id=3512

Comments


Diary Archives