Welcome to WebmasterWorld Guest from

Message Too Old, No Replies

Understanding the "Crawl Rate" Explanation in WMT



5:35 am on Dec 19, 2008 (gmt 0)

5+ Year Member


This is regarding the "Crawl Rate" functionality of Google webmaster tools. They have explained it as follows

"The crawl rate affects the speed of Googlebot's requests during the crawl process. It has no effect on how often Googlebot crawls your site. Google determines the recommended rate based on the number of pages in your site."

My Question :

How do you differentiate between "speed of requests" and "how often Googlebot crawls the site". If the speed of requests increases that don't you think that my website is getting crawled pretty often?

Pls help me out with is.



6:05 am on Dec 19, 2008 (gmt 0)

"Speed of requests" refers to the speed of the crawl--i.e., the interval between each page request during a crawl. In some cases, limiting the crawl speed might be desirable to avoid overloading a busy server. In other words (to use purely a hypothetical example), a weekly crawl might take 30 minutes or an hour, depending on the chosen crawl speed, but it would occur once a week in either case.

Featured Threads

Hot Threads This Week

Hot Threads This Month