homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
Forum Library, Charter, Moderators: mademetop

General Search Engine Marketing Issues Forum

Is a hidden link to a blocked internal page bad?
to catch bad bots

10+ Year Member

Msg#: 4523032 posted 9:13 pm on Nov 27, 2012 (gmt 0)

I'd like to start building a list of bad bots that are hogging up server resources. One idea I found while researching is to create a new directory, and block it in robots.txt. Then put a hidden link to the new directory on my homepage. Something like:

<a style="display: none;" href="/doNotEnter/"></a>

User-agent: *
Disallow: /doNotEnter/

No human will see the link, and no reputable bot will follow it. Therefore anything that does enter that directory can be scrutinized and perhaps added to a block list.

Sounds like a good idea, I'm just concerned that search engines will see the hidden link to a blocked directory and assume something fishy is going on. Any thoughts?



WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

Msg#: 4523032 posted 9:52 pm on Nov 27, 2012 (gmt 0)

I have a similar setup on sites that are doing fine. I'm using a file that is blocked rather than a directory but it is in many places on the sites and is doing what it is supposed to do. I do not use the
style="display: none;"
but it is a nofollowed link with onclick set to false.
Global Options:
 top home search open messages active posts  

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved