Robots Text -need to have one?

[glengara]

Up to now I've assumed the lack of one implied an open door to to Ms. Bot.

Since it's apparently the first thing looked for, am I being rude by not offering one?

[korkus2000]

I didn't have one for a while and it didn't hurt me with google. I think it is important on other engines.

[engine]

It's not essential.

Primarily, it's to "control" the spiders visiting your site. If you do not want the spiders to index a particular page or subdirectory, use a robots.txt file in conjunction with meta noindex (where appropriate).

Some pesky spiders do need controlling. ;)

Take a look here for controlling spiders with robots.txt. [webmasterworld.com]

[angiolo]

The only annoyance is the 404 message in your logs.

Ditto for favicon.

[Brett_Tabke]

[webmasterworld.com...]