Welcome to WebmasterWorld Guest from

Forum Moderators: bill

Message Too Old, No Replies

Microsoft Follows Up On Cloud Service Outage

4:58 pm on Sep 21, 2011 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
votes: 343

Microsoft Follows Up On Cloud Service Outage [windowsteamblog.com]
So, what happened? A tool that helps balance network traffic was being updated and the update did not work correctly. As a result, configuration settings were corrupted, which caused a service disruption.

We determined the cause to be a corrupted file in Microsoft’s DNS service. The file corruption was a result of two rare conditions occurring at the same time. The first condition is related to how the load balancing devices in the DNS service respond to a malformed input string (i.e., the software was unable to parse an incorrectly constructed line in the configuration file). The second condition was related to how the configuration is synchronized across the DNS service to ensure all client requests return the same response regardless of the connection location of the client. Each of these conditions was tracked to the networking device firmware used in the Microsoft DNS service.

After restoring service, we have identified two streams of work to drive specific service improvements around monitoring, problem identification, and recovery. Along with these service improvements, Microsoft is focused on further hardening the DNS service to improve its overall redundancy and fail-over capability.

Earlier story Microsoft Cloud Services Outage Appears to be DNS Related