tangor - 5:40 am on Oct 3, 2011 (gmt 0)

people are likely to want to "crawl" the site by doing lots of search queries rather than link-based crawling.

Separate issue... I have a few sites like this where I offer site-based search (no bots allowed, run my own index). Keeps it all "in-house".