tedster
msg:3801383 | 2:58 am on Dec 6, 2008 (gmt 0) |
Apparently this custom crawl rate is only available for root level sites - not sites not hosted on a large domain like Blogspot. Google says they assign special settings in that type of case.
|
rash
msg:3801558 | 1:59 pm on Dec 6, 2008 (gmt 0) |
i am not on blogspot and it won't allow me to change the crawl rate. The message I get is your site has been assigned special crawl rate settings. Perhaps this is due to the fact that my site is relatively new
|
azn romeo 4u
msg:3801676 | 7:05 pm on Dec 6, 2008 (gmt 0) |
I put my crawl rate at 200% more than the default. I saw that it goes up to 900%.
|
AhmedF
msg:3801692 | 7:55 pm on Dec 6, 2008 (gmt 0) |
Pretty cool how it is dynamic. One of my sites has a peak of 10 pages per second, whereas another has a peak of 0.5 pages per second.
|
kamikaze Optimizer
msg:3801783 | 11:55 pm on Dec 6, 2008 (gmt 0) |
Has anyone noticed how long it takes for the change to take effect? I increased one of my sites but I do not see a change in the Google Bot spider rate on site.
|
kamikaze Optimizer
msg:3801805 | 1:22 am on Dec 7, 2008 (gmt 0) |
I can answer my own question now, it appears to take about 2 hours. This is a great feature. What would make it even better would be if Google would allow throttling by time of day. I would love to throttle up to the max rate on the over night hours and bring it back to normal during the day. That would be great.
|
drall
msg:3802050 | 7:13 pm on Dec 7, 2008 (gmt 0) |
I dont understand the value in this... "However, setting it to higher value than the default won't improve your coverage"
|
SEOPTI
msg:3802055 | 7:27 pm on Dec 7, 2008 (gmt 0) |
Set it to 900% and the yo-yo effect will probably go wild :)
|
fsmobilez
msg:3851534 | 3:54 pm on Feb 17, 2009 (gmt 0) |
>>>Set it to 900% and the yo-yo effect will probably go wild :) You mean we have to avoid it or what ? What is this crawl rates actually means as i increased it to 0.5 requests per second but no change in site traffic , but yes in google wmt i can see lot of network unreachable errors , and hosting company site is up 24 hours So plz help me what to do Should i change crawl rate? What positive and negative effects 0.5 request per second have ? and where i can get full details regarding this feauter If u can give details here i will be thankful.
|
|
