Forum Moderators: open

Message Too Old, No Replies

Googlebot just taking robots and /

Everyone on this shared server in the same boat

         

CosmicDan

5:52 pm on Apr 20, 2003 (gmt 0)

10+ Year Member



Between the 17th and 18th, Deepbot grabbed the robots.txt and homepage from every virtual site on my hosts server. The Freshbot has poked every 3 days as usual this month, but it too has only taken 1 page and not crawled deeper. Freshie was doing deeper crawls until March 31st, but has only done the "sniffle" since then.

I saw some other posts of people who seemed to be experiencing similar things. I was hoping that folks in that boat would post here with the date of the deepbot "sniffle" and then again if they actually got crawled.

I'd like to know if some moron on the shared host has managed to get the whole server clobbered somehow.

heini

10:26 am on Apr 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



CosmicDan, I don't think that's got anything to do with the server. From what I saw and read it looks much more like the bots are not working up on regular speed for some reason.