Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: phranque
Just look to these numbers, it's crazy.
Google crawler is 100 times more aggressive than Yahoo's crawler.
How do I reduce the frequency? Also take a lot of bandwidth. This is this month only! 5 Days. Also the reach from google to my site is only double than the one from yahoo...
Googlebot 14268+191 223.37 MB
Yahoo Slurp 144+6 31.30 MB
MSNBot 25+30 839.49 KB
This are stats generate by AWSTATS
Any ideas how to I reduce the frequency?