Do we have to use robots.txt on every website?

for robots.txt issues

[masood82]

i have a question is it essential to use robots.txt on every website for engine crawlers or not

[topr8]

robots.txt is completely unneccesairy.

many, many sites do not have one.

[it has a purpose of course, but you do not need it]

[Stay123]

Search engines look at millions of web pages to come up with search results. They do this with what we call "search engine spiders." This makes sense - spiders crawling around on the Web. But another word for them is "robots" because they are simply unmanned programs gathering data automatically. With the use of robots.txt file search engine better know your website.

[Marcia]

Putting up a robots.txt - even a blank one - prevents getting a 404 every time a crawler tries to fetch it and it isn't there.

[topr8]

>>Putting up a robots.txt - even a blank one - prevents getting a 404 every time a crawler tries to fetch it and it isn't there.

indeed and ditto fav.ico