Forum Moderators: goodroi
i found a website, where someone says this:
"If you don't have any restricted area on your site (like me) you don't need a robots.txt file. Don't even put an empty robot.txt file as some robots interpret a blank robots.txt file as meaning "don't crawl any pages on this website."
he is wrong, right? because i have an empty robots.txt on my site and i want all sites to get spidered.
I dont know about the blank file. But if you want to use the file and tell all search engines that your site is wide open, I suggest that you copy paste this in it.
User-agent: *
Disallow:
Here is a great ressource. [searchengineworld.com...]
[edited by: Macguru at 8:30 am (utc) on Mar. 5, 2003]