Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Restrict Specific Domain Names from Being Indexed

Using the robots.txt file on Google to restrict certain domains...



12:23 am on Oct 29, 2003 (gmt 0)

10+ Year Member

Hi there all :)

I have a website at www.olddomain.com that was listed on Google. I have recently been able to purchase a better domain name that has great keywords - www.newdomain.com.

I would like Google to spider the site, but only list the www.newdomain.com domain as I do not want to get penalised for spamming multiple addresses. Basically we use www.olddomain.com for print and www.newdomain.com for web promotion.

Any ideas?

Thanks in advance.

- Ben Slater

[edited by: heini at 9:43 am (utc) on Oct. 29, 2003]
[edit reason] Please don't post real urls or domain names / Thanks! [/edit]


12:45 am on Oct 29, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

If you're on Apache, you can use mod_rewrite to redirect robots.txt requests to different files based on the requested domain name using {HTTP_HOST}. On other servers, you could use a script to do essentially the same thing.

So, one domain appears to have a robots.txt that allows spidering, and the other one disallows spidering.

See [webmasterworld.com...]



1:36 am on Oct 29, 2003 (gmt 0)

10+ Year Member

Hmmmm, thanks for the advice Jim. I will follow up with my webhost to see if they can help...it's good to go in with a little bit of knowledge so I don't look like a complete lamer ;)

- Ben


Featured Threads

Hot Threads This Week

Hot Threads This Month