Welcome to WebmasterWorld Guest from 23.22.46.195

Forum Moderators: phranque

Best way to prevent bots from following links

anything better than redirect.xyz exluded in robots.txt

   
12:57 pm on May 20, 2004 (gmt 0)

10+ Year Member



If I'd like to make sure no bot is following links my (maybe a little simple) solution would be:

- use a redirect-file like red.cgi
- put it in some folder like domain.com/admin/red.cgi
- exlude folder /admin/ in robots.txt

Question is if this wouldn't draw any unappreciated attention...

Would you use anything more sophisticated?

5:30 am on May 21, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Do you mean legitimate bots, or spambots? For legitimate bots, the robots.txt should work, or meta robots nofollow.

For spambots, I'd try javascript links. No guarantees, though. Some of them will pull out anything that looks like a url.

6:07 am on May 21, 2004 (gmt 0)

10+ Year Member



>> legitimate bots

yes.

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month