Welcome to WebmasterWorld Guest from 54.204.198.71

Forum Moderators: goodroi

Message Too Old, No Replies

Disallowing Indexed files in Google

     
9:48 pm on Dec 20, 2006 (gmt 0)

New User

10+ Year Member

joined:Feb 11, 2006
posts:26
votes: 0


Hi
Lets say I have a few hundred paged in regular index and few thousand in Supplemental index. I dont need Google to index those page, is it ok to put them in robots.txt (with wildcards).

If so what will happen to those in regular index? will the drop out immediately or move to supplemental and eventually go out?

any experience with the same? if yes how long it takes? Any effect to rankings, traffic (which is why I am doing this in the first place)

Please let me know. Thanks in Advance.

Leo.

10:14 pm on Dec 20, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 23, 2004
posts:78
votes: 0


Yes, that is exactly how robots.txt is to be used.

Just remember that Google isn't the only online source of your indexed pages.

2:44 am on Dec 21, 2006 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3154
votes: 129


hi leo,

i think we might want to take a second, step back and try to figure out what you want. generally if you are trying to improve rankings and traffic, link development and not robots.txt is the way to go. also make sure you pages have sufficent unique content.

you mentioned you are doing this because of rankings and traffic. why do you think robots.txt will help you? is it because they are just duplicates? also why dont you want google to index the pages it labeled as supplemental?

8:29 pm on Dec 22, 2006 (gmt 0)

New User

10+ Year Member

joined:Feb 11, 2006
posts:26
votes: 0


Hi
I want to remove those pages from the index becuase they have little or no content (empty customer reviews etc).

I know it may hurt my page rank on some pages becuase the "real" pages will loose internal link "votes" in google's eyes. But it will atleast cleanup my site index. Based on other threads, it is probably true that Google is penalizing sites becuase of emptpy pages and what not.

Thanks
Leo

2:01 pm on Dec 23, 2006 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3154
votes: 129


If you have empty pages and these pages will never gain good content, then it is wise to get rid of these pages.

If you use robots.txt for this you may lose some link popularity. If you use htaccess to do some 301 redirects you can save the link popularity. For me it would depend on how many pages and how many links from other sites are pointing to these pages.

cheers