It appears anything hosted by Westhost has been down since about 3:30pm eastern time (roughly three hours now), and they are working on the problem.
3:22 pm on Mar 1, 2010 (gmt 0)
I think I mentioned that a few posts ago about who WestHost is, or is not. "Cogentco's web servers and the websites in California, on 10TB. Which is owned by the same company that owns Westhost." They actually have two vendors. Cogentoco was the one hit. Right on about INERGEN ..LOL we are just assuming it was the electronics fire suppression system and not the sprinkler system which is really what sounds like activated. That would create some excitement.
4:57 pm on Mar 1, 2010 (gmt 0)
Although the gases in Inergen do not damage electronics by themselfs, the release of it can. the gas mix relies on dilution of the oxygen in the air from 21% to about 10%. To do this you have to add about the same volume of gas to the room as already present. Without quick pressure release systems in the server room this will increase the pressure significantly upto 1 bar+. If you have ever opened a computer and looked at your harddisk, you will see that most harddisks are covered with a thin sheet of metal. A pressure surge of 1 bar will cause a force of about 100 kg (200 pounds) on that sheet of metal which will probably bend and damage the heads and other delicate mechanics inside.
Many older data centers were designed for use with Halon type fire suppression systems. Halon gases do not replace the oxygen in the air, but they catch the radical oxygen atoms to prevent them to bind with flamable substances. Much less gas is needed and there is not a significant pressure increase in the server room. Halons are now forbidden because they damage the Ozon layer around the Earth. Older data centers which moved from Halon to Inergen fire suppression systems may not have the adequate pressure release systems in place to prevent mechanical damage due to the pressure surge.
5:07 pm on Mar 1, 2010 (gmt 0)
I thought the problem with Halon (we used to have it in AF) is it does replace oxygen and our brains don't pick up on the lack of oxygen..we had several people killed that way..maybe the O2 binding is the issue. Good point about over pressure. That would do it. We always had to have an air pack for the electronics rooms.
6:26 pm on Mar 1, 2010 (gmt 0)
New problem may be emerging of hackers getting access to multiple servers during all this confusion. Several sites of mine are getting the Reported Attack Site message in Firefox.
"This web site at www.samplesite.com has been reported as an attack site and has been blocked based on your security preferences."
Had a look at a couple of my sites and, yes, there've been changes to files.
Thought it was just the sites on my dedicated server, but it appears not. Sorted all of those out but the VPS accounts seem affected as well. Different servers, different passwords.
WH, in typical style, is pretending the problem lies elsewhere. Google's message says 116 sites on one of those shared servers are affected.
You heard it here first.
2:53 am on Mar 4, 2010 (gmt 0)
My site was down twice today for about a 1/2 hour each time and 3 client sites were down for a short time yesterday. I don't have it set up to send me a report everytime a client site goes down or I may have seen a lot more were down (I just happened to be working on the sites at the time).
3:34 am on Mar 4, 2010 (gmt 0)
Yes, we had an ecomm site go down for 2-3 hours yesterday, along with another on the same server. These were sites that were relatively unaffected by the original outage. So, something is still going on.
On a related note, I noticed that they were unable to recover data on some of the crashed servers, but were optimistic they could recover the data on the backup drives. Ouch!
The lack of transparency continues to be troublesome...
7:32 pm on Mar 4, 2010 (gmt 0)
My site was down most of the night and won't be up till tomorrow morning along with another site I manage on another server. Something to do with RAID.
no email either. I'm contacting clients through another email. - can't even get any spam! Help! I'm having withdrawals!
4:34 pm on Mar 18, 2010 (gmt 0)
Today is the final straw. All of our sites have had severe lag all morning and there's been no response from support.
Westhost seems incapable of consistently delivering the service that I've paid for, so it's time to go elsewhere.
I'd appreciate PMs with personal recommendations of VPS hosts.
4:49 pm on Mar 18, 2010 (gmt 0)
Yes my main email on 1 server is down again.
I am almost done moving over everything to my new host but it is a lot of work..... don't take a move lightly if you manage a lot of sites.
I am going with a local company and I am going there every day to swap out up a backup drive and keeping it in my safe deposit box.
I have learned that you can't put your fate in the hands of others and you especially can't leave the fate of your clients in the hands of another. I have also learned I like to have a door to knock on when things go wrong.