Microsoft Outlook and Hotmail outages caused by overheating datacentre

Posted on May 11, 2013 at 7:04 pm

Microsoft has said that outages on its Outlook and Hotmail services were caused by an overheating datacentre that took itself offline to stop problems spreading further.

The incident occurred on Wednesday and left scores of customers without access to their email accounts.

In a blog post written by vice president of test and service engineering, Arthur de Haan, the firm gave more insight into the cause of the issue.

“On 12 March, in one physical region of one of our datacenters, we performed our regular process of updating the firmware on a core part of our physical plant. This is an update that had been done successfully previously, but failed in this specific instance in an unexpected way,” he said.

“This failure resulted in a rapid and substantial temperature spike in the datacenter. This spike was significant enough that it caused our safeguards to come in to place for a large number of servers in this part of the datacenter.”

He explained this meant access to mailboxes housed on the servers was suspended and this also stopped other pieces of infrastructure acting as a failover to allow access.

“Once the safeguards kicked in on these systems, the team was instantly alerted and they immediately began to get to work to restore access,” de Haan continued.

“Based on the failure scenario, there was a mix of infrastructure software and human intervention that was needed to bring the core infrastructure back online.

“Requiring this kind of human intervention is not the norm for our services and added significant time to the restoration.”

De Haan apologised for the outage and said he hoped the explanation would give customers some understanding of the situation that faced the firm.

The incident underlines that issues around cloud computing continue to hit the headlines and may cause some businesses to continue to be wary of the model for hosting business critical applications.

Posted in Cloud Hosting