Msg#: 3811008 posted 5:35 am on Dec 19, 2008 (gmt 0)
This is regarding the "Crawl Rate" functionality of Google webmaster tools. They have explained it as follows
"The crawl rate affects the speed of Googlebot's requests during the crawl process. It has no effect on how often Googlebot crawls your site. Google determines the recommended rate based on the number of pages in your site."
My Question :
How do you differentiate between "speed of requests" and "how often Googlebot crawls the site". If the speed of requests increases that don't you think that my website is getting crawled pretty often?
Msg#: 3811008 posted 6:05 am on Dec 19, 2008 (gmt 0)
"Speed of requests" refers to the speed of the crawl--i.e., the interval between each page request during a crawl. In some cases, limiting the crawl speed might be desirable to avoid overloading a busy server. In other words (to use purely a hypothetical example), a weekly crawl might take 30 minutes or an hour, depending on the chosen crawl speed, but it would occur once a week in either case.