| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|Best way to prevent bots from following links|
anything better than redirect.xyz exluded in robots.txt
If I'd like to make sure no bot is following links my (maybe a little simple) solution would be:
- use a redirect-file like red.cgi
- put it in some folder like domain.com/admin/red.cgi
- exlude folder /admin/ in robots.txt
Question is if this wouldn't draw any unappreciated attention...
Would you use anything more sophisticated?
Do you mean legitimate bots, or spambots? For legitimate bots, the robots.txt should work, or meta robots nofollow.
>> legitimate bots
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved