- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Googlebot not respecting custom crawl rate
g1smd - 1:27 am on Jun 18, 2011 (gmt 0)
If you are setting this through robots.txt, have you used the WMT feature to fetch the robots.txt file just in case there is something in there that it can't read?
Thread source: http://www.webmasterworld.com/robots_txt/4325733.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com