Welcome to WebmasterWorld Guest from 54.145.144.101

Forum Moderators: goodroi

Message Too Old, No Replies

Disallowing Indexed files in Google

   
9:48 pm on Dec 20, 2006 (gmt 0)

5+ Year Member



Hi
Lets say I have a few hundred paged in regular index and few thousand in Supplemental index. I dont need Google to index those page, is it ok to put them in robots.txt (with wildcards).

If so what will happen to those in regular index? will the drop out immediately or move to supplemental and eventually go out?

any experience with the same? if yes how long it takes? Any effect to rankings, traffic (which is why I am doing this in the first place)

Please let me know. Thanks in Advance.

Leo.

10:14 pm on Dec 20, 2006 (gmt 0)

10+ Year Member



Yes, that is exactly how robots.txt is to be used.

Just remember that Google isn't the only online source of your indexed pages.

2:44 am on Dec 21, 2006 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



hi leo,

i think we might want to take a second, step back and try to figure out what you want. generally if you are trying to improve rankings and traffic, link development and not robots.txt is the way to go. also make sure you pages have sufficent unique content.

you mentioned you are doing this because of rankings and traffic. why do you think robots.txt will help you? is it because they are just duplicates? also why dont you want google to index the pages it labeled as supplemental?

8:29 pm on Dec 22, 2006 (gmt 0)

5+ Year Member



Hi
I want to remove those pages from the index becuase they have little or no content (empty customer reviews etc).

I know it may hurt my page rank on some pages becuase the "real" pages will loose internal link "votes" in google's eyes. But it will atleast cleanup my site index. Based on other threads, it is probably true that Google is penalizing sites becuase of emptpy pages and what not.

Thanks
Leo

2:01 pm on Dec 23, 2006 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



If you have empty pages and these pages will never gain good content, then it is wise to get rid of these pages.

If you use robots.txt for this you may lose some link popularity. If you use htaccess to do some 301 redirects you can save the link popularity. For me it would depend on how many pages and how many links from other sites are pointing to these pages.

cheers