Disallowing Indexed files in Google

[leo11877]

Hi
Lets say I have a few hundred paged in regular index and few thousand in Supplemental index. I dont need Google to index those page, is it ok to put them in robots.txt (with wildcards).

If so what will happen to those in regular index? will the drop out immediately or move to supplemental and eventually go out?

any experience with the same? if yes how long it takes? Any effect to rankings, traffic (which is why I am doing this in the first place)

Please let me know. Thanks in Advance.

Leo.

[TXGodzilla]

Yes, that is exactly how robots.txt is to be used.

Just remember that Google isn't the only online source of your indexed pages.

[goodroi]

hi leo,

i think we might want to take a second, step back and try to figure out what you want. generally if you are trying to improve rankings and traffic, link development and not robots.txt is the way to go. also make sure you pages have sufficent unique content.

you mentioned you are doing this because of rankings and traffic. why do you think robots.txt will help you? is it because they are just duplicates? also why dont you want google to index the pages it labeled as supplemental?

[leo11877]

Hi
I want to remove those pages from the index becuase they have little or no content (empty customer reviews etc).

I know it may hurt my page rank on some pages becuase the "real" pages will loose internal link "votes" in google's eyes. But it will atleast cleanup my site index. Based on other threads, it is probably true that Google is penalizing sites becuase of emptpy pages and what not.

Thanks
Leo

[goodroi]

If you have empty pages and these pages will never gain good content, then it is wise to get rid of these pages.

If you use robots.txt for this you may lose some link popularity. If you use htaccess to do some 301 redirects you can save the link popularity. For me it would depend on how many pages and how many links from other sites are pointing to these pages.

cheers