Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Is setting the crawl-delay rate to 5 seconds can help or hurt SEO?

         

virtualreality

2:42 am on Apr 20, 2015 (gmt 0)

10+ Year Member Top Contributors Of The Month



I have a site that's performing well in organic search. My host recommended to set the crawl-delay rate to 5 seconds. I an concerned to do that because I dont know if this can affect SEO in a negative way? Please advise - do you think this is a good or bad idea?
Thank you!

lucy24

6:03 am on Apr 20, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Google ignores the Crawl-Delay directive in robots.txt. So if that's what you're asking about it will have no effect at all.

Five seconds seems like a long time. If one googlebot visit every four seconds is placing undue strain on your server resources, maybe it's time to go host-shopping.

goodroi

12:17 pm on Apr 20, 2015 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Crawl delay isn't an issue. It is a sign of a potential bad host. Having a bad host can impact SEO rankings. Spend some money to get reliable and fast hosting.