robot.txt question

[Awilliamson]

Is it recommended to add a line to robot.txt to have googlebot NOT cache the site contents? Does this affect the crawl at all?

Thanks in advance.

[korkus2000]

I have heard that this could be a red flag, but who really knows. I don't think there is a real problem with it as far as the crawl goes. This site prevents google caching. I have seen people stop caching more lately.

[Netizen]

The robots.txt is for disallowing a spider from your site, not about caching - that is the "robots" meta tag.

[mars9820]

For not caching I always use the META tag on the page itself.

<META NAME="GOOGLEBOT" CONTENT="NOARCHIVE">

Hope this helps.

[DaveAtIFG]

Netizen is correct. See the robots.txt tutorial [searchengineworld.com] and Google's suggestions [google.com] section B, item 2 re: caching.