Welcome to WebmasterWorld Guest from 54.196.208.6

Forum Moderators: mack

Message Too Old, No Replies

I am about to ban msnbot

crawl delay

     
11:03 pm on Sep 21, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 6, 2002
posts:1825
votes: 21


They don't respect the crawl delay in robots.txt

Are you experiencing the same?