google and robots.txt - do I need one?

[cheater copperpot]

Simple question:

Will google find me if i dont have a robots.txt file? or do i need one? I dont have any file named robots.txt on my server now.

Thanks

[Brett_Tabke]

No, the Robots Exclusion Standard is to (you guessed it) exclude robots - not include. Welcome to the board.

[jdMorgan]

CC,

However, if you monitor your site logs, you can save yourself from having to wade through a lot of
404-Not Found errors by uploading at least a blank robots.txt file, as recommended by GoogleGuy.

This prevents your server from sending a 404 response code back to robots which request robots.txt,
and from logging that error response. But since the file is empty, it doesn't exclude any robots.

More info here [robotstxt.org].

Jim

[idiotgirl]

No, you don't need to have a robots.txt file, and including one does not guarantee any bots you want to exclude will request and/or recognize it. However, you can use it as a tracking tool if you habitually review your access logs. Also, if robots.txt is requested by a bot and not found, it will generate 404 errors in your access log. If you don't mind the 404's in your access logs, aren't tracking robots.txt accesses, and don't wish to exclude any robots.txt-minding bots, it really isn't necessary at all. If you choose to include one, on the other hand, there is a ton of posts on WebmasterWorld (and throughout the web) for more information :)

<added>jdMorgan pretty much said already this while I was typing this!</added>

[cheater copperpot]

ok cool guys thanks alot. I guess I will just go ahead and throw a blank one in for now as recommended by GG.