homepage Welcome to WebmasterWorld Guest from 50.17.27.205
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
how to block indexing of root domain, but still allow sub-domain
how to block indexing of root domain, but still allow sub-domain
chazeo




msg:3765554
 5:41 pm on Oct 14, 2008 (gmt 0)

I currently have domain.com being redirected to sub-domain.domain.com via htaccess file (301). However pages in domain.com are still being indexed by Google.

How do I stop domain.com pages from being indexed, without risking the sub-domain.domain.com pages? Robots.txt?

 

goodroi




msg:3766059
 1:04 pm on Oct 15, 2008 (gmt 0)

It would probably be easier and also better for the link popularity of the new domain if you broaden your 301 redirect. You are probably only redirecting index.html. You can change the 301 into a generic one that will match all urls at the old domain and redirect them to the new domain.

chazeo




msg:3766144
 2:49 pm on Oct 15, 2008 (gmt 0)

Thanks for the reply! The old domain is a wordpress blog, the sub-domain is a forum, so there are no matching URLS. Would removing the blog/db help? Any URL from the blog redirects to the main page of the forum index...

goodroi




msg:3767011
 12:50 pm on Oct 16, 2008 (gmt 0)

If you are already redirecting all the blog urls into the forum index page then you do not need to worry about blocking the pages. It will just take some time for the spiders to revisit your site and update their index.

chazeo




msg:3767113
 2:52 pm on Oct 16, 2008 (gmt 0)

It's been over a year and blog pages are still showing up in SERPS

g1smd




msg:3770361
 1:20 pm on Oct 21, 2008 (gmt 0)

Use the Live HTTP Headers extension for Firefox to confirm that when you ask for any of the old URLs, that the first response from the server is a 301 redirect to the correct URL on the new site.

If the redirect is working fine, then you have done as much as you can. Google can take more than a year to remove old URLs sometimes. If you have a redirect in place then anyone clicking those links is taken to the new content, so no damage is done.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved