Forum Moderators: open
Is there a way to NOT have a web page crawled/indexed? I have a copyright notice page that is linked to from other pages. I've had a quite a few visitors enter my site on this page. It's probably not the best page for a "first impression" since it deals with my terms of use, etc. Can I put something in my robots.txt file so google doesn't index it?
Thanks!
Or whatever the appropriate file name is. Make sure to validate your robots.txt file [searchengineworld.com] each time a change is made just to make sure that everything is okay.
I've had a quite a few visitors enter my site on this page.
The above statement is what would prevent me from blocking that page. If you've got a respectable number of visitors coming in through that page, its no big deal. As long as they have access to navigation that will lead them to the information they were searching for.