Welcome to WebmasterWorld Guest from 54.163.100.58

Forum Moderators: goodroi

robots.txt

used for site with 2 urls

   
7:36 pm on May 6, 2003 (gmt 0)

10+ Year Member



I have a site that we have recently moved to a new URL. We want it to remain under the old URL for about a year because we have marketing brochures that have the old URL in them. These brochures are being phased out over the next year.

I don't want Google to think I'm spamming them with the site under multiple urls.

Should I write a robots.txt to tell spiders (google's in particular) not to spider the site under the old url?

If so, how would that code appear in the robots.txt file?

If not, is there a better way to deal with it?

12:05 am on May 7, 2003 (gmt 0)

10+ Year Member



The only way google recognizes a URL change without thinking that you're doing sneaky redirects is to use a 301 moved permanently redirect. You can do this in your .htaccess file. It has nothing to do with the robots.txt file.

In your case, if you add a 301 to olddomain, all your brochure visitors to olddomain will get instantly redirected to newdomain, so you won't have to tell them later which domain to go to. Googlebot will also get instantly redirected to newdomain, and they'll stop spidering olddomain.

You should also contact all your inbound links and get them to change to newdomain.

2:11 am on May 7, 2003 (gmt 0)

10+ Year Member



And if you are wondering about the syntax to use for your .htaccess file it is:-

redirect permanent / [mynewsite.com...]

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month