Welcome to WebmasterWorld Guest from 22.214.171.124
What would the basic file look like without excluding anything.
User-agent: * Disallow:
A great and very easy to understand tutorial is here: [searchengineworld.com...]
And welcome to the board!
Let's see if I undertand,
User-agent: * Disallow: /forum/
Would just keep the spiders off my forum right? Everything else would be found.
where does the file go?
in the root directory? of the WWW root?
my dir structure is....
/ www cgi-bin images [all html content]
should i put robots.txt here with the HTML content? or one level up?
Test with: [mydomain.com...]
Become a Pro Member