robots.txt

[indodpendo]

If you dont have a robot.txt the search engines will just index your site, Why have the robots.txt file

[phidentity]

You may not want certain/all search engines to see certain/all pages on your website.

Jon

[SEOMike]

Robots.txt will help keep SEs out of directories you don't want indexed. For example, someone who sells his images might not want google's imagebot to pull all of his images out of a directory and list them where they can be freely downloaded.

[jdMorgan]

And having any robots.txt file --even a blank one-- will prevent a lot of 404-Not Found errors in your logs, thus making it easier to identify real problems.

Jim