Some new pages not being indexed

[gsmith]

On my site I cover a number of businesses, each having its own website. I had a writer produce my own original text describing each business, consisting of one paragraph containing around 120 words. GWT reports that there are no duplicate content issues. Nevertheless, many of these pages have not been accepted into the index, while some others have.

The material for these one-paragraph descriptions was derived mainly from information available on the businesses own website, so the word "rehash" might be appropriate.

Could this explain why many of these pages have not been accepted into the index?

[tedster]

Just to clarify, are you saying that these missing pages are not showing up in the site: operator results?

Additionally, has googlebot even spidered these new pages? There's a big differenece begtween spiderd but not inlcuded, and never spidered.

[gsmith]

Just to clarify, are you saying that these missing pages are not showing up in the site: operator results?

Additionally, has googlebot even spidered these new pages? There's a big differenece begtween spiderd but not inlcuded, and never spidered.

They're not showing up using the site: operator.

I've submitted a Sitemap file, which has been accepted without error. and googlebot has been through my site.

On my GWT Sitemaps page it displays

Total URLs: 316
Indexed URLs: 63

Interestingly, the site: operator shows 46 - 50 pages, but I know that this issue has been discussed elsewhere.

Thanks for your interest.

[tedster]

How new is the site, and how recently were these new pages added? Have you got some backlink love from around the web?

[gsmith]

The site is brand new, and I do have some backlinks, but only to the home page. The home page shows up on page 6 for the most relevant keyword.

I first submitted the sitemap two and a half weeks ago, and have been resubmitting the sitemap as I get new input from the writer since then. While attempting to answer your question I realized that the only business-specific pages that have been indexed are those that Google knew about from before I started submitting the sitemap, but certainly not all of them. The order that they're being added is in accordance with the order that internal links to them were added; so it looks like it's a simple matter of patiently waiting.

[gsmith]

PS: I added a sitemap file because almost all of those business-specific pages are accessible to the user only via some Web 2.0 stuff. There are no links to most of those pages, either internal or from outside.

Does this mean that Google might decide not to index them? Googlebot has been through all of the pages in my sitemap file, so it definitely knows that they're out there.

[tedster]

Is this some kind of Ajax scripting where the content only gets served to the visitor after an on-page action makes a new server request?

Whatever the technology is, I'd suggest running Xenu Link Sleuth on the site and seing if it can find your new pages.

[wizboy]

I can't believe the spiders can read and differentiate a re-written paragraph of content. That seems very difficult to do with a computer.