Page is a not externally linkable
pleeker - 7:58 am on Dec 30, 2006 (gmt 0)
If the bot doesn't find a robots.txt, it assumes it's okay to crawl anything and everything it finds under that domain. If you're okay with that, then no worries. :)
Your impression is correct that the purpose of robots.txt is just to control what obedient bots will and won't crawl.