I think I mentioned that a few posts ago about who WestHost is, or is not. "Cogentco's web servers and the websites in California, on 10TB. Which is owned by the same company that owns Westhost." They actually have two vendors. Cogentoco was the one hit. Right on about INERGEN ..LOL we are just assuming it was the electronics fire suppression system and not the sprinkler system which is really what sounds like activated. That would create some excitement.
Although the gases in Inergen do not damage electronics by themselfs, the release of it can. the gas mix relies on dilution of the oxygen in the air from 21% to about 10%. To do this you have to add about the same volume of gas to the room as already present. Without quick pressure release systems in the server room this will increase the pressure significantly upto 1 bar+. If you have ever opened a computer and looked at your harddisk, you will see that most harddisks are covered with a thin sheet of metal. A pressure surge of 1 bar will cause a force of about 100 kg (200 pounds) on that sheet of metal which will probably bend and damage the heads and other delicate mechanics inside.
Many older data centers were designed for use with Halon type fire suppression systems. Halon gases do not replace the oxygen in the air, but they catch the radical oxygen atoms to prevent them to bind with flamable substances. Much less gas is needed and there is not a significant pressure increase in the server room. Halons are now forbidden because they damage the Ozon layer around the Earth. Older data centers which moved from Halon to Inergen fire suppression systems may not have the adequate pressure release systems in place to prevent mechanical damage due to the pressure surge.
I thought the problem with Halon (we used to have it in AF) is it does replace oxygen and our brains don't pick up on the lack of oxygen..we had several people killed that way..maybe the O2 binding is the issue. Good point about over pressure. That would do it. We always had to have an air pack for the electronics rooms.
My site was down twice today for about a 1/2 hour each time and 3 client sites were down for a short time yesterday. I don't have it set up to send me a report everytime a client site goes down or I may have seen a lot more were down (I just happened to be working on the sites at the time).
Yes, we had an ecomm site go down for 2-3 hours yesterday, along with another on the same server. These were sites that were relatively unaffected by the original outage. So, something is still going on.
On a related note, I noticed that they were unable to recover data on some of the crashed servers, but were optimistic they could recover the data on the backup drives. Ouch!
The lack of transparency continues to be troublesome...
I am almost done moving over everything to my new host but it is a lot of work..... don't take a move lightly if you manage a lot of sites.
I am going with a local company and I am going there every day to swap out up a backup drive and keeping it in my safe deposit box.
I have learned that you can't put your fate in the hands of others and you especially can't leave the fate of your clients in the hands of another. I have also learned I like to have a door to knock on when things go wrong.