How important is robot.txt

Importance of robot.txt for a website

[trraju]

Hi

i was just going through my website stats and i have seen few things which is may be unacceptable by few searchengines.

data :

HTTP Error codes
HTTP Error codes* Hits Percent Bandwidth
404 Document Not Found 1421 67.2 % 0
302 Moved temporarily (redirect) 605 28.6 % 115.42 KB
301 Moved permanently (redirect) 72 3.4 % 20.33 KB
206 Partial Content 16 0.7 % 1.39 MB

so here if we see the Document Not Found is getting hit for more than 1421 in this month and wanted to know how important is it for websites and what we need to do to rectify thease errors.

I also wanted to know how to make a robot.txt which should be compatible with all search engines like google, yahoo and adsense too.

regards
trraju

[DanA]

You may find an answer here
[webmasterworld.com...]
or there
[webmasterworld.com...]

[trraju]

thanks ... will check them :)