Forum Moderators: open
I have some Flag Links (feedback, spam, etc..) links and I want only live people to be able to follow these links, not robots. Is this possible?
<a href="www....com">Only Real-Live People</a>
<META NAME="robots" CONTENT="noindex,nofollow">
place this between the <head> - </head> part of your page. It should work if the search engines follow meta robots tags.
<!-- Google off -->
<!-- Google on -->
I have seen it on more than one of the TV networks online news divisions.
<!-- Google off --><!-- Google on -->
My guess is this is directed at the news.google.com bot.
Search Webmaster World for all need to know about it.
:)
zoo
This is a good way to scan your access logs for bots accessing files that you don't want and then banning their ip/agents from returning.
This is a bit premature -- META tags can only be read by crawler that read THAT page, where as links that were disallowed by META may have been used on other sites. This means banning bots for accessing links not disallowed in robots.txt but just METAs is not really fair thing to do.
At link level it's: rel=nofollow
NoFollow is really used as NoRank, ie to avoid assigning weight to that link pointing to another page. They really picked misleading name, Google itself states:
"From now on, when Google sees the attribute (rel="nofollow") on hyperlinks, those links won't get any credit when we rank websites in our search results."
This does NOT mean link won't be followed, just it won't get credit.
It may take a day or so. Good spiders obey. Ones that do not, will probably not obey anything.
Since the link simply reloads the current page, only appends a couple of variables onto the url, javascript may be the only solution!
[searchtools.com...]
There is some better explanations here. WIth the robots.txt file you can tell certain bots not to index certain pages.
The solution I came up with may be overkill but works great. I have a script which identifies if the 'Visitor' is a robot or known spider. If it is the three links I DO NOT WANT available to bots are not displayed!
I need something on the link level however. I want the bots to include or spider the page, just not follow a select number of links on the page...
<a rel="nofollow" href="webpage.html">
Not sure how many bots other than Google support this
<a rel="nofollow" href="webpage.html">Not sure how many bots other than Google support this
NoFollow value or rel attribute is a misleading name not designed to prevent bots from following URLs. It was designed to give hint to indexers to adjust (to zero) weight of the link in their link analysis algorithms.
Its incorrect naming gives impression that link won't be followed, but this is not what this value was designed for. Here is what Google said about it:
"From now on, when Google sees the attribute (rel="nofollow") on hyperlinks, those links won't get any credit when we rank websites in our search results."
