- - Search Engines
- -- Search Engine Spider and User Agent Identification
- ---- msn bot has lost its mind
lucy24 - 1:02 am on Sep 5, 2013 (gmt 0)
Do they honor the Crawl-Delay directive? I know Google doesn't-- you have to set it in wmt-- but I'm ### if I can find the area in Bing wmt that analyzes your robots.txt.
Thread source: http://www.webmasterworld.com/search_engine_spiders/4607276.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com