I own a busy website about a specific tourist destination and we recently got whacked in the SERPS for our main keyword term. Obviously, I'm exploring the possible reasons to hopefully restore our rankings and I'm currently investigating several theories on our current SERPS demise.
As well as the main website we have a busy forum with half a million posts and 7,500 registered users. Inevitably, due to the social nature of forums and the light hearted happy subject material (going on vacation) we get a lot of off topic posts. While we do have a specific sub forum for this kind of discussion we have a ton of threads related to subjects that have nothing to do with the principal website material - sporting discussions, birthday wishes, word association games, jokes etc.
While Google spidering the forum adds a lot of pages to the index and increases our page count, many of these pages are irrelevant to the search terms I'm aiming for.
If Google is indexing and listing this kind of irrelevant forum content (which far outnumbers the pages of real content) is this going to dilute my real content pages worth, and thus harm my SERPS? or is it better to have the maximum number of pages in the index possible regardless of content?
Obviously, many of our forum posts are on topic and do bring in some SE traffic. I'm wondering though, whether to block the whole forum with my robots.txt as Google may be getting confused about what we are really about.