Forum Moderators: open

Message Too Old, No Replies

How do I disallow spiders?

I want to know what code to put on a page to not allow spiders to crawl.

         

Mitch

5:02 pm on May 20, 2002 (gmt 0)

10+ Year Member



I already know about having a robots.txt file in the root directory of a site which tells spiders not to crawl certain areas. But, what code can I put on a web page that would disallow spiders from crawling it? Thanks for any feedback.

lazerzubb

5:03 pm on May 20, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I guess you're looking for this
[robotstxt.org...]