Forum Moderators: goodroi

Message Too Old, No Replies

i have a subdomain I dont want indexed

is robots.txt the answer?

         

chrisandsarah

12:34 pm on Aug 8, 2004 (gmt 0)

10+ Year Member



Hi
I dont know how, but google has picked up my sub domain i created as a test site for my main site, so now i have a mirror of my main site in google that i fear may get me penalised
Is creating a robot.txt the answer and telling all bots not to index?
I'm worried that this will stop them crawling my main site.
E.g www.testarea.mydomain.com is my sub domain,
www.mydomain.com is my main.
Can someone tell me what lines i need in my robots text, and should the instructions go on my main site, or subdomain?

Also, ive recently been dropped from yahoo. Would this problem have caused it? e.g yahoo found my sub domain and classed it as a mirror of my main?

many thanks for any help

encyclo

12:57 pm on Aug 8, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If the test area is a mirror of your primary domain, and it has been index by accident, then yes a robots.txt file disallowing everything wouldn't be a bad solution.

However, if the test area is showing up in the SERPs, then you should really do a 301 redirect for everything to your main domain, create a new test area, and to avoid any problems in the future, you should password-protect the new test area so it absolutely cannot be indexed.

chrisandsarah

1:06 pm on Aug 8, 2004 (gmt 0)

10+ Year Member



Hi. Many thanks for that.
Ive just tried looking on the web but hope someone could answer me quicker in here.
What lines should i put in my .htaccess to redirect everything in my subdomain to my main site?

many thanks for any help

and can anyone answer my previous questions about how to tell the bots not to crawl my sub domain but still allowing them to crawl my main.

much appreciated

Birdman

1:21 pm on Aug 8, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Something like this in your root .htaccess:

RedirectMatch permanent (.*)$ http:/www.domain.com$1

As far as keeping the bots out, use robot.txt in the new test subdomain or password protect the subdomain as mentioned before.

Birdman