Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: goodroi
If you have an META tag instruction for robots, do you need a robots.txt?
At the very least, you should put an empty robots.txt to cut down on 404s on your site.
Using META tags allows you to control how robots index or follow links on your pages.
Using robots.txt restricts access to files (text and binary), as well as directories.