Forum Moderators: goodroi

Message Too Old, No Replies

Blocking Google

I need to block google from my index page

         

loner

1:32 pm on Mar 31, 2007 (gmt 0)

10+ Year Member



I'm moving a site to an existing domain name- in pieces. Both sites have been around for 8-10 years and are causing problems coming up for the same keyword search in google.

I only want to block google from spidering the very top index.html of the top directory of the old site. Since the move will be taking awhile, I'd still like for google to spider the rest of the site.

Although I've been web-stuff since '95, I'm old, muddled, dazed and confused, and never bothered with a robots.txt before. I got no secrets, but it has become an issue and I don't want to screw this up for the rest of the 6,000+ pages I have on the site.

Would this be what I need?

User-agent: google
Disallow: /index.html

encyclo

6:56 pm on Mar 31, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Your example would not exclude the home page when not called as /index.html.

You basically have two choices - the first is to force a redirect to http://example.com/index.html when http://example.com/ is called, then exclude index.html.

The second and much easier method is to use a meta noindex element on the page:

<meta name="robots" content="noindex">

loner

6:25 am on Apr 1, 2007 (gmt 0)

10+ Year Member



Thanks, that should do it.

gtate

4:25 pm on Apr 2, 2007 (gmt 0)

10+ Year Member



I have a website I do not want to be found by the search engines. I therefore inserted thge code "<meta name="robots" content="noindex,nofollow">" into my templated pages, but find it is still being picked up. Help!