Forum Moderators: phranque
User-agent: Googlebot
Disallow: /
If you want Googlebot to crawl some pages but not others, say to restrict it from one particular directory, use this directive:
User-agent: Googlebot
Disallow: /directorytorestrict/
Basic information about robots.txt can be found at robotstxt.org [robotstxt.org]. Specific information on how Google has implemented robots.txt can be found here [google.com].