Welcome to WebmasterWorld Guest from

Message Too Old, No Replies

Understanding the "Crawl Rate" Explanation in WMT

5:35 am on Dec 19, 2008 (gmt 0)

Junior Member

5+ Year Member

joined:June 13, 2008
votes: 0


This is regarding the "Crawl Rate" functionality of Google webmaster tools. They have explained it as follows

"The crawl rate affects the speed of Googlebot's requests during the crawl process. It has no effect on how often Googlebot crawls your site. Google determines the recommended rate based on the number of pages in your site."

My Question :

How do you differentiate between "speed of requests" and "how often Googlebot crawls the site". If the speed of requests increases that don't you think that my website is getting crawled pretty often?

Pls help me out with is.


6:05 am on Dec 19, 2008 (gmt 0)

Senior Member

joined:July 3, 2008
votes: 0

"Speed of requests" refers to the speed of the crawl--i.e., the interval between each page request during a crawl. In some cases, limiting the crawl speed might be desirable to avoid overloading a busy server. In other words (to use purely a hypothetical example), a weekly crawl might take 30 minutes or an hour, depending on the chosen crawl speed, but it would occur once a week in either case.