Is Google spidering creating a bandwidth issue for you?
3:04 am on Jul 27, 2003 (gmt 0)
Between Google and Inktomi my server is getting spidered like crazy. I was wondering if anyone else is seeing a lot of spidering and how it is affecting your server performance.
[edited by: beggers at 5:33 am (utc) on July 27, 2003]
5:05 am on Jul 27, 2003 (gmt 0)
Yeah, I'm just getting spidered like crazy. I typically serve up 3,000 pageviews a day, but my logs have been 7,000 a day or even more. The spiders are just grinding through my site non-stop: Google, Teoma, Inktomi, and some I don't recognize. Google's definitely coming by once a day.
I haven't checked to see what percentage is from Adsense, though.