When Your Host Toasts Your Domains

GBot knocks, but can't get in...

[PatrickDeese]

I have over 100 domains (belonging to clients and myself)hosted at what was perceived to be a good and reliable hosting company (and recommended by a number of WW'ers) - however - suddenly every site on their server's is "drying up" in Google - tons of URL only listings - pages not cached, and sites with Dmoz/G Directory listings not showing the directory icon in the GTB...

First pass with their tech support: "Maybe google is banning your domains". I say I don't think so - I have 90+ other domains hosted with other companies, and they are a-okay.

Second pass with their tech support: "It's not us - we're not blocking anyone". So I write [e-mail] - and they say that the domains I mentioned are not penalized - but weren't accessible in the last crawl(s).

Third pass with their tech support: "Hey Google says they can't crawl sites hosted by you". Tech support doesn't believe me - so I write back to G and say - pretty please - can you check example.com via a crawler IP - my host says they aren't doing anything to block Google. Google says - something is blocking gBot.

Is it time to move these domains - are these guys being coy with me - or are they clueless.

My guesstimate is that in total, so far, at least 2500 - 3000 pages from 100+ domains have been dropped due to this issue.

How long do you give a host to make things right?

[isitreal]

They told me there may have been a bandwidth saving measure in place, by blocking GoogleBot.

Amazing, I'm glad I dumped weak hosters permanently, I did my research and have had zero problems with mine, and could find zero problems online about them before I switched all my stuff to them, I'd say the netcraft hoster uptime ratings are not that useless, mine rated number 4 at the time, beaten only by a few other mega type places like yahoo, so really they were number 1, go freebsd and your odds jump, happily a lot of people go with others which leaves the good ones free to remain good.

Look for this: privately held, not a part of a corporation that will squeeze them to cut costs, active supporters of the free/opensource stuff, which means they put their money where their mouths are, run only serious os's, no windows, ever, and are big enough to be able to handle growth and traffic surges. The list shrinks rapidly, there might be a tiny handful of companies that are this good in the US.

[flex55]

Tim / Blue Gravity,

I got to tell you, this is just amazin. I'm shocked to learn what happened, and if I were hosted with you I'd immediately run away to a host that understands the web better.
These are the kinds of mistakes that you just cant do.

[sh0ck]

This is not a mistake we have done.

We have not done anything on our network to prevent google from accessing us and I have verified with my upstream providers with traceroutes and pings that EVERY google ip is available from EVERY ip on our network.

I challenge google to say this is not them blocking us again because it clearly is and we have no idea why and they seem to be providing misinformation to our customers.

Google if you are watching give me proof that one of your ips cannot get to one of mine! This is insane. I want traceroute logs showing where it stops if you cant get to us because I can get to you from everywhere on my network. New ips, old ips it doesnt matter. Please explain why this is happening because it has nothing to do with us.

--tim

[mipapage]

FWIW - I host with these folks and have yet to have any concrete problems of this sort.

I can understand patrickdeese's need to move, money is on the line and he can't wait around to see what the issue is.

I have spoken on the phone with Tim and he has been very helpful with this issue (with any issue - I've never had customer service like I have with thses people), has done a lot of work (himself and with the various groups that they work with) to try and figure it out. Nothing has turned up.

At this point it would be helpful for Google to work with them to solve this issue, IMHO.

[grandmac]

I read that if you host with GoDaddy that new pages are slow to get indexed. My new pages are being slow but they say it shouldn't be. Does anyone remember seeing this info and where? Or are you familiar with this company to know something about slow indexing with them?
Thanks,
Carol

[isitreal]

If you host with godaddy, oh, that's painful, the best answer is don't host with them, expect problems bad service etc. Get a real webhoster and you'll be much happier.

[webtress]

I just took over a site with a different host than mentioned above and am experiencing the same as patrick. The pages are in the index but only with the www.sitename.com for all the pages, no descriptions and the cache is old pre-April. I've resubmitted waited and still nothing. I suspect it is host related but don't know how to test for it. The user agent report only shows browsers and operating systems, I have not seen any reference to the gbot at all, for that matter no robots,spiders or crawlers.

How can I tell if the host is turning them away?
Is there a list of known host or IP ranges that google can't spider?

[webtress]

Well ask you you will receive. I replaced the original robots.txt with the one I use and Kazzam, the gb2 came through.

[g1smd]

Did this get fixed?