Forum Moderators: goodroi
it is called a robots.txt (s)
Your log file will show a 404 because the search engine spider has requested the robots.txt file but it was unavailable, you don't have to have the file in your root directory but by having even a blank file will stop the errors from showing in the log file.
I personally use the robots.txt file, I think it is better not to give a spider a 404 but to serve the file, as for affecting SE ranking it won't affect the results but as an overall seo package it's important not to serve 404's.
Robots.txt Tutorial
[searchengineworld.com...]