| Welcome to WebmasterWorld Guest from 184.108.40.206 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
crawl, spider, and robot.
I've got 3 robots that visit my site often. I'm assuming it's competition running page analysis on me. How can I block these?
Unknown robot (identified by 'crawl')
Unknown robot (identified by 'spider')
Unknown robot (identified by 'robot')
You can only bloxk them by the IP they come from in your htaccess file. You need to have Mod-rewrite installed to do so - can be very tricky!
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved