Page is a not externally linkable
-- Google SEO News and Discussion
---- Pages are indexed even after blocking in robots.txt
shaunm - 7:07 am on Sep 3, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/google/4490125.htm
Thank you so much!
But it doesn't look like a snippet, it just looks how other pages are displayed in Google's search results. I have have come through this snippet stuff for other websites though.
|Google then constructs a title and snippet for the URL just from references rather than by crawling the page directly. |
Oops, I wouldn't have done that if you hadn't informed me. Thanks a ton!
|And remember to change your robots.txt file so you now ALLOW googlebot to crawl the page. Unless they crawl, they won't ever read the robots meta tag. |
Are you sure that even though I may block a web page using noindex meta tags, the page will still be indexed if the URL has been included in the SITEMAP?!? Because I have never heard of this before. Can you give me some references or share your personal experiences? THanks
|If the URL is in your sitemap, the page will be crawled. |
Thanks for that mate! I will keep those things in mind as well :)
Thanks again, ted :)
Thank you for replying on my post and for the provided URL reference :)
Brought to you by WebmasterWorld: http://www.webmasterworld.com