Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- GWT URL Error (on my Blogger.com site)


Susan_Moskwa - 11:10 pm on Oct 10, 2007 (gmt 0)


Hi folks--
I work with the Google webmaster tools team and can clarify some of these issues for you:

All Blogger blogs have a robots.txt file automatically created for them (add "/robots.txt" to the end of your blog's URL and you'll see yours). These files all disallow the /search directory, which is part of the path when you're viewing all of your blog posts that have a particular label. Disallowing crawlers from this section of your blog basically keeps them from crawling and indexing the same blog post in multiple places (on its permalink URL *and* under each of its labels), which reduces potential problems with duplicate content.

adfree and silverbytes, it sounds like this is the cause of the errors you're seeing in webmaster tools. There's no way to get rid of them, since Blogger doesn't let you edit your robots.txt file, but they're not something you need to worry about (since those URLs were disallowed deliberately).

Drew_Black, it sounds like the 301 redirect issue you're talking about may be unrelated to Blogger? If there's a 301 redirect from page A to page B and page *B* is blocked by a robots.txt file, then page *A* will show a "Restricted by robots.txt" error in webmaster tools (there's a blog post from September '06 on the Google Webmaster Central blog with more details about this). But the opposite shouldn't be true (page A is blocked but page B shows the error). I believe I've found your thread in our Webmaster Help Group (a search for [chaosunlimited destination] in our Help Group returns your question, right?); could you post an example URL there so that we can look into the issue further? Thanks!


Thread source:: http://www.webmasterworld.com/google/3468726.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com