NoIndex - Do I neeed Robots.txt AND meta

[wheelie34]

I have just noticed something on an old site of mine, it has a directory making up a large portion of it, each category has a suggest your site link which points to add_url.php which doesnt really have unique content but the usual form to add site details. As theres many categories for the same page to work on (only change per page is the breadcrumb so the script knows which category to put it in)

I dont think it's wise to allow it to be counted as a page so would like to NOT be indexed, I have put in this tag

<meta name="robots" content="noindex">

Is that enough or should I also put

Disallow: /add_url.php

In the robots.txt file, thanks

[Robert Charlton]

I dont think it's wise to allow it to be counted as a page so would like to NOT be indexed, I have put in this tag

<meta name="robots" content="noindex">

Is that enough or should I also put

Disallow: /add_url.php

In the robots.txt file, thanks

wheelie34 - I prefer the robots meta tag, used by itself. Google will index the url of a page blocked by robots.txt if it finds a link to that page elsewhere. Google won't index references to a page blocked by the robots meta.

But, if you block spidering of the page with robots.txt, then the spiders won't see the robots meta tag.

I've not looked at your robots.txt syntax, but I assume you've got it right.

[wheelie34]

Thanks Robert, I thought the meta tag would be enough but its always best to ask.

Cheers

[g1smd]

The noindex meta tag still allows spiders to spider the page, but the details of that page will not appear in the SERPs at any time.

The robots.txt exclusion stops spiders from spidering the page, but the URL can still appear in the SERPs if the spiders discover that URL in a link on some other page elsewhere.

Those usually appear as URL-only entries in the SERPs but Yahoo can even construct a title for the entry by using the anchor text from one of those incoming links to your site.