Content pages are spidered but erratically not indexed

Why would some HTML pages no longer be indexed/cached?

[chap0209]

Since last summer (Bourbon update?), I noticed that Google no longer indexes up to half of the pages of my site (though they are still always spidered). In other terms, there is no cached version and only the URL appears in a site: search.

The number of indexed pages varies erratically one week after the other, from half of the pages to almost all of them.

This a PR 5 site, exising for more than 3 years, with a fair number of backlinks, and until this summer all the pages were indexed by Google. The URLs are simple, with no arguments (e.g. .../mypage.htm). There are 76 pages with highly rich and original content on the way financial markets work.

Has any of you experienced this? Usually, non-indexed pages have URL arguments. I still have to find a PR 5 site with a lot of simple HTML pages spidered but not indexed.

The only specific thing that I see is that my site uses a lot of JavaScript to generate the navigation bar on top and on the left (this was originally intended to ease maintenance, the navigation tree being declared once and for all in a .JS file).
Everything that GG indexes is in the final page, I only add content for navigation, so I do not think this can be considered as "cloaking". Could this still be a problem?

[Thez]

I have something like 700,000+ html pages (ie, the pages are dynamic, but but rewritten to .html). Site has pr 5 (has been 6), i've submitted all pages via sitemaps, and googlebot crawls a lot. Yet only 184k pages indexed. Site has been in google for 7 years. Yes, all of these pages have unique content.

All my competitors have majority if not all pages indexed but I don't. It kinda pisses me off :/ But it's been a month or less since I submitted the sitemaps, so maybe soon there will be a change. I sincerely hope there will be. It would propably double or even triple my traffic :\

[chap0209]

Thanks Thez, I can also imagine the difference in the traffic! Please tell us in the future if Google sitemaps have made any difference - I read however that it could only help getting more spidered pages (but not indexed).

Have you or someone else noticed any drop in the number of indexed pages since last summer, just as I did?