Welcome to WebmasterWorld Guest from 54.145.235.72

Forum Moderators: mack

I am about to ban msnbot

crawl delay

   
11:03 pm on Sep 21, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



They don't respect the crawl delay in robots.txt

Are you experiencing the same?

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month