Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
site on profile
its a good question. In my instances, the only content I disallow is content not relevant to the site (ie contact us/feedback/buy this) sort of thing.
You have to consider that if you exclude certain parts of your site, that you will prevent the bot from navigating your whole site, and it will affect your overall navigation (as far as certain search engines and page rank is concerned)
im sure one of the experienced ones in here will be able to chip in. My largest site is about the quarter of the size of yours and its pretty much left open to spidering.