| 2:58 am on Dec 6, 2008 (gmt 0)|
Apparently this custom crawl rate is only available for root level sites - not sites not hosted on a large domain like Blogspot. Google says they assign special settings in that type of case.
| 1:59 pm on Dec 6, 2008 (gmt 0)|
i am not on blogspot and it won't allow me to change the crawl rate. The message I get is your site has been assigned special crawl rate settings. Perhaps this is due to the fact that my site is relatively new
|azn romeo 4u|
| 7:05 pm on Dec 6, 2008 (gmt 0)|
I put my crawl rate at 200% more than the default. I saw that it goes up to 900%.
| 7:55 pm on Dec 6, 2008 (gmt 0)|
Pretty cool how it is dynamic. One of my sites has a peak of 10 pages per second, whereas another has a peak of 0.5 pages per second.
| 11:55 pm on Dec 6, 2008 (gmt 0)|
Has anyone noticed how long it takes for the change to take effect? I increased one of my sites but I do not see a change in the Google Bot spider rate on site.
| 1:22 am on Dec 7, 2008 (gmt 0)|
I can answer my own question now, it appears to take about 2 hours.
This is a great feature.
What would make it even better would be if Google would allow throttling by time of day. I would love to throttle up to the max rate on the over night hours and bring it back to normal during the day. That would be great.
| 7:13 pm on Dec 7, 2008 (gmt 0)|
I dont understand the value in this...
"However, setting it to higher value than the default won't improve your coverage"
| 7:27 pm on Dec 7, 2008 (gmt 0)|
Set it to 900% and the yo-yo effect will probably go wild :)
| 3:54 pm on Feb 17, 2009 (gmt 0)|
>>>Set it to 900% and the yo-yo effect will probably go wild :)
You mean we have to avoid it or what ?
What is this crawl rates actually means as i increased it to 0.5 requests per second but no change in site traffic , but yes in google wmt i can see lot of network unreachable errors , and hosting company site is up 24 hours
So plz help me what to do
Should i change crawl rate?
What positive and negative effects 0.5 request per second have ?
and where i can get full details regarding this feauter
If u can give details here i will be thankful.