Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: open
22.214.171.124 - - [01/Mar/2004:12:30:03 -0700] "GET /forum/viewforum.php?f=3& HTTP/1.0" 200 18718 "-" "Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com; htXp://www.alltheweb.com/help/webmaster/crawler"
"That's odd herb. Are you sure your robots.txt is valid?"
The robots file is and has been valid.
Ink has spidered the site ignoring the restricted areas for the past year. This just started two days ago.
Yesterday it was visiting about once an hour and grabbing only the forums.
Received a message from the folks at Yahoo and they are reviewing the situation.