| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|How to stop Google from indexing a page?|
I need to block the following page alone from Google Index:
| 9:33 am on Jul 8, 2014 (gmt 0)|
I need to block the following page alone from Google Index through Robots.txt:
h t t p://stores.example.com/BlueWidgets
I don't want to make Google to stop crawling the following pages.
h t t p://stores.example.com/BlueWidgets/Mobile-Phones/Model1
h t t p://stores.example.com/BlueWidgets/Mobile-Phones/Model2
Need your suggestion? Thanks in Advance!
[edited by: engine at 9:48 am (utc) on Jul 8, 2014]
[edit reason] obfuscated [/edit]
| 12:47 pm on Jul 8, 2014 (gmt 0)|
welcome to WebmasterWorld, sathishvaiha!
if you want to "stop Google from indexing a page" you should use meta robots noindex element.
using robots.txt will exclude a bot from crawling but is not an indexing directive.
| 6:39 pm on Jul 8, 2014 (gmt 0)|
Technical answer: Googlebot does recognize the "Allow" directive, so what you want to do is theoretically possible. You Disallow the directory, and then Allow its individual pages.
But a "noindex" meta is probably a better way to go-- especially when you're talking about rewritten URLs where the directories don't physically exist.
| 4:19 am on Jul 9, 2014 (gmt 0)|
Awesome replies! Thanks @phranque & @lucy24.
| 2:55 am on Sep 12, 2014 (gmt 0)|
Sadly, you will have to let the G see the page to see the directives. the only way to keep G from seeing a page is to password protect it.
Aside: don't rely on noindex to keep G from reading the page. they have to to see noindex. Catch-22
Have to ask: Why do you want to disallow G for that page? (General response, do not be too specific)
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved