Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: goodroi
I do see that when a spider comes a crawlin' they seem
to find my robots.txt file just fine.
The question is: where the heck is that file? The tutorial
says to put it in the root folder...does that mean the
robots.txt file should be included in the html of the index
P.S. I use dreamweaver 4 if that matters.
Oh, and beware of text editors, make your robots.txt file with Notepad or BBEdit (or maybe even use Pico to create the file on your server if it is some kind of Unix).
If you have uploaded your robots.txt file and still in doubt, use the Robots.txt Validator at SearchEngineWorld [http].