Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Canonical Tag vs. Block in Robots.txt


TheMadScientist - 11:40 pm on Apr 26, 2011 (gmt 0)


I'd use a 301 or my second choice would be a canonical link element in the head + noindex.

See this thread is you think you have your robots.txt correct and GoogleBot is crawling the URL(s) ... There could be an 'interpretation issue' [webmasterworld.com...]

Important note from vanessafox:
Googlebot follows the line directed at it, rather than the line directed at everyone.

It makes sense to me from the examples given in the thread.

ADDED: noindex to 'what I would do'.

Robots.txt would be something I used as a 'last resort' ... You lose any and all link juice in all search engines with it, but with a 301 or canonical you retain it in some way whether all engines interpret and use the the canonical link info or not.


Thread source:: http://www.webmasterworld.com/google/4302977.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com