| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Best way to prevent bots from following links|
anything better than redirect.xyz exluded in robots.txt
| 12:57 pm on May 20, 2004 (gmt 0)|
If I'd like to make sure no bot is following links my (maybe a little simple) solution would be:
- use a redirect-file like red.cgi
- put it in some folder like domain.com/admin/red.cgi
- exlude folder /admin/ in robots.txt
Question is if this wouldn't draw any unappreciated attention...
Would you use anything more sophisticated?
| 5:30 am on May 21, 2004 (gmt 0)|
Do you mean legitimate bots, or spambots? For legitimate bots, the robots.txt should work, or meta robots nofollow.
| 6:07 am on May 21, 2004 (gmt 0)|
>> legitimate bots
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved