is there a reason to use a robots.txt

[thorthedestroyer]

If I create a robots.txt will it do a better job at getting all pages indexed then simply having the following listed on every page:

<meta NAME="robots" CONTENT="index,follow,all">
<META NAME="GOOGLEBOT" CONTENT="INDEX, FOLLOW">

Thanks so much in advance.

[robotsdobetter]

The robots.txt is used to block robots. The robots.txt and the Meta tags won't help you more pages spidered. If you want to get more web pages spidered get more backlinks.

[duckxtales]

what are backlinks again?

[Dijkgraaf]

Backlinks are links pointing to your site from other sites.

[stapel]

To answer the question in your subject line, however, there is a reason to use a robots.txt file: Responsible robots check for it, so not having one will generate unnecessary 404 errors.

So even if your robots.txt file is empty (that is, you have nothing sensitive that you don't want good bots to stumble in to, so you've nothing to protect), having the file on the server will reduce your 404 error log.

Just a thought....

Eliz.