homepage Welcome to WebmasterWorld Guest from 54.161.155.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Global "no follow" and discretionary "follow"
dummiped




msg:3826360
 7:23 pm on Jan 14, 2009 (gmt 0)

Hi, I just found out that Wikipedia implements "no follow" because of the existence of deadlinks (red links) due to internal hyperlinking.

As my website is also using the Mediawiki software like Wikipedia and since it is a one-year old site with a lot of deadlinks (red links), I wonder whether I can have a global "no follow" in the robot.txt file, followed by the following:

<meta name="Robots" content="index,follow">

on each article page, as soon as I manage to clear all the red links for each page.

 

goodroi




msg:3827083
 2:34 pm on Jan 15, 2009 (gmt 0)

I do not know about mediawiki but I do know that robots.txt can not apply nofollow to links. Robots.txt is a voluntary protocol that helps webmasters tell robots which files and folders should be accessed and which should not be accessed. If you block a file using robots.txt the search engine bots will not access it so they will not see the individual robots meta tag.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved