Welcome to WebmasterWorld Guest from 126.96.36.199
What would the basic file look like without excluding anything.
User-agent: * Disallow:
A great and very easy to understand tutorial is here: [searchengineworld.com...]
And welcome to the board!
Let's see if I undertand,
User-agent: * Disallow: /forum/
Would just keep the spiders off my forum right? Everything else would be found.
where does the file go?
in the root directory? of the WWW root?
my dir structure is....
/ www cgi-bin images [all html content]
should i put robots.txt here with the HTML content? or one level up?
Test with: [mydomain.com...]
Las VegasOctober 6-8, 2015