I have 160 page site that normally gets a few pages (10-20) crawled twice a week, and a full crawl once a month.
In the last 4 hours, GoogleBot has hit every page 3 or more times and robots.txt 60 times. Anybody else seeing this craziness?
Brett_Tabke
2:27 pm on Apr 5, 2005 (gmt 0)
Ya, there were alot of people that reported a strong Gbot presence last week.
lost in space
7:03 pm on Apr 5, 2005 (gmt 0)
GBot is killing one of my servers with this also, they've been hitting one of my sites 30,000 times per day for about the last 10 days, it keeps causing my CPU to overheat, I think they're trying to catch my server on fire :)
jnmconsulting
7:19 pm on Apr 5, 2005 (gmt 0)
are we sure it is an actuall google bot, best way to tell is by IP not the useragent! I had this happen a few weeks ago and found it was an ip that was assigned to comcast cable and not google.
walkman
7:21 pm on Apr 5, 2005 (gmt 0)
I think before and after a major update this happens.
BillyS
7:45 pm on Apr 5, 2005 (gmt 0)
I think before and after a major update this happens.
My experience exactly.
itisgene
8:30 pm on Apr 5, 2005 (gmt 0)
So, is it before or after? I didn't care much about the Allegra anyways...but who knows? My sandboxed sites may get out if there is another update.