Welcome to WebmasterWorld Guest from

Message Too Old, No Replies

why Gbot accessed only Robots.Txt but not site pages



9:59 am on Apr 2, 2005 (gmt 0)

10+ Year Member

On 24th all my websites published on a number of different servers - (all at one host i.e.MCi having a unique IP number (all within the 193.131.180.x range) had a unique Google accessibility problem. FOr all these sites, the access from google for anything other than robots.txt suddenly ceased for over a period of about 7 hours. The requests for existing pages reduced over time and only robots.txt was requested All of our sites which were more than 30 in number, were affected on GOOGLE around the same time while MSN search etc were not affected at all.

The sites were accessed correctly on 23rd Feb by the Googlebot from the following Data Centres:

[,...] [,...] [,...] [,...] []

Only the robots.txt were accessed on 24th Feb by the Googlebot from the following Data Centres:

[,...] [,,...] [,...] [,...] [,...] [,...] [,...] []

Although I had fixed my site by adding a robots.txt file to the root of each site. And now wwe have 'service as usual'. But, the thing I'm concerned about is what could have been the possible reason for Googlebot's weird behaviour, which had happened for the first time in almost
I'm really concerned about sorting this problem at my site, since these are our very important sites.
Would be really interested to know:
-Why this happened? Was it us or them?!
Thanks in advance for any views!-Deeps


Featured Threads

Hot Threads This Week

Hot Threads This Month