Welcome to WebmasterWorld Guest from 188.8.131.52
i'd like to solicit comment on a not so hypothetical site, long established site with:
2) a storefront which serves pages from the categories, but the page information has an embedded iframe with the all the (non commercial) information provided with an inline frame (iframe).
3) Over the years, with code changes, directory renaming, etc etc.. there are LOTS of rewrite/redirect rules
4) Google has managed (not surprisingly, given the changes and things done over the site's history) to index most of the store's pages, but only a handful of the "real content" pages, and when that's done (indexing of "real content" pages), it's only a few pages for each category (usually the list or table of contents for the category). BTW category content pages have relevant title, description, etc.
Another option? is to start over with a new site and gradually link or rewrite everything to the new site?
I think the key, in addition to technical precision, is not to change urls unless you absolutely can't see another way. In those cases, do a study to see if you really need a 301 redirect to take care of pages that are getting good search traffic or have strong backlinks. Only set up a 301 in those cases, and let the rest of your old urls return a 404.
Just make sure you've got a good linking structure. That alone helps Google sort out your revised site. Without a pile of 301 redirects to chekc for trust issues, Google can handle the revised site pretty quickly.
And speaking of a study, I'd really look closely at the traffic to those category/brand pages,including whether those visitors convert. That kind of page can really be a useful thing for some sites.