| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Subscribe to WebmasterWorld|
crawl, spider, and robot.
| 9:16 pm on Nov 14, 2003 (gmt 0)|
I've got 3 robots that visit my site often. I'm assuming it's competition running page analysis on me. How can I block these?
Unknown robot (identified by 'crawl')
Unknown robot (identified by 'spider')
Unknown robot (identified by 'robot')
| 4:10 am on Nov 16, 2003 (gmt 0)|
You can only bloxk them by the IP they come from in your htaccess file. You need to have Mod-rewrite installed to do so - can be very tricky!
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
© Webmaster World 1996-2014 all rights reserved