Forum Moderators: Robert Charlton & goodroi
The site has been indexed OK, and used to show a fresh date on every page every few days. The site also ranked well before this, but as of nearly two weeks ago, all of the spammy pages have lost their cache link and no longer show any fresh dates. There is a lot of content duplication within the spammy pages too.
The normal content pages still have a cache link, and still show a new fresh date in the SERPS, every few days.
It looks like the spam has been detected and Google is heading towards wiping it from their SERPs.
.
My guess is that if your site has already been around for a while, that some sort of penalty is about to hit it. Use Xenu LinkSleuth to check all your internal links, and WebBug to check that you have a proper 301 redirect from non-www to www for all pages of the site. Make sure that every title and meta description is different from that on every other page of the site, and that each tag accurately reflects the content of the page that it is on.
If the site is very new, then other factors may be at work (but do the above checks anyway).
If you are doing the exclusion via the robots.txt file, and these pages are new, then it is possible that Google hasn't yet seen the exclusion in that file or acted on it.
Note: For old pages that are already indexed by Google, adding an exclusion in the robots.txt file will NOT cause those pages to be unidexed. They will stay indexed forever, because Google indexed them at a time when you allowed them to be indexed.