Stop that SE Bots

[Bud_Bundy]

Hi,

How do I exclude a page from being crawled by SE bots?

Thanks.

[RonPK]

To prevent a page from being crawled, create a file called robots.txt and put it in the root of your site. Its contents:

User-agent: *
Disallow: /path/to/your/file.html

More on robots.txt at [searchengineworld.com...]

To prevent the contents of a page from being indexed, use the robots meta tag. It can be configured to have the spider not index the page you've put it on, but to follow the links:

<meta name="robots" content="noindex, follow">