Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Google refuses to crawl a site

many incoming links, no googlebot

5:11 pm on Jun 14, 2004 (gmt 0)

10+ Year Member

There are links from pages as high as PR5 and low as PR0 pointing to this site's front page. There have been no hits from the crawler, even on robots.txt, in over a month. The MSN bot visits very often.

I've searched, and most of threads like this relate to new websites, or new webmasters learning that websites need incoming links for google to visit. This page had been listed in the SERPs for months and crawled almost daily until early May.

I'm pretty sure this is unusual. Has anyone else seen this happen? Is there anything I can do to bring back the crawler?

8:22 pm on Jun 14, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

change your host and put some new content on your home page.
11:55 pm on Jun 14, 2004 (gmt 0)

10+ Year Member

How would a new host help? Posts I've seen here suggest google doesn't care about the site's IP, even if it is shared hosting. I think GoogleGuy said it himself somewhere.

If IP's matter anyway, would asking my host to move be to another box or IP help?

3:19 am on Jun 15, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

a new host may help in a couple of scenarios.
1) you have been identified as part of a incestuous nest.Moving host may just help you get outside that loop.
2) there is a problem with your hosts setup similar to the one godaddy had 6 months back which disallows search engine crawling.
Putting new content on your page may help if you have a competitor cloning your site.
Of course these are longshots but worth trying.
You just may also have a penalty for hidden text or something or google may have flagged you for spam of some sort and you maybe on your way out.:(
3:39 pm on Jun 15, 2004 (gmt 0)

10+ Year Member

I don't think its a spam related thing. I'll try changing hosts next month. Thank-you!
3:44 pm on Jun 15, 2004 (gmt 0)

10+ Year Member

I think there were some issues with GoDaddy hosting a few months back that caused problems with Googlebot and many sites were not being crawled.

In instances like that, changing hosts would probably fix the issue.