homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

used for site with 2 urls

 7:36 pm on May 6, 2003 (gmt 0)

I have a site that we have recently moved to a new URL. We want it to remain under the old URL for about a year because we have marketing brochures that have the old URL in them. These brochures are being phased out over the next year.

I don't want Google to think I'm spamming them with the site under multiple urls.

Should I write a robots.txt to tell spiders (google's in particular) not to spider the site under the old url?

If so, how would that code appear in the robots.txt file?

If not, is there a better way to deal with it?



 12:05 am on May 7, 2003 (gmt 0)

The only way google recognizes a URL change without thinking that you're doing sneaky redirects is to use a 301 moved permanently redirect. You can do this in your .htaccess file. It has nothing to do with the robots.txt file.

In your case, if you add a 301 to olddomain, all your brochure visitors to olddomain will get instantly redirected to newdomain, so you won't have to tell them later which domain to go to. Googlebot will also get instantly redirected to newdomain, and they'll stop spidering olddomain.

You should also contact all your inbound links and get them to change to newdomain.


 2:11 am on May 7, 2003 (gmt 0)

And if you are wondering about the syntax to use for your .htaccess file it is:-

redirect permanent / [mynewsite.com...]

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved