Page Not Indexed, Cached Version Shows as Homepage

[simplesimon]

I've been having problems keeping a page indexed for a few months now. It is an internal page with a single link from the homepage. It had done well in a semi-competitive SERP for over a year until it dropped from the index 3 or 4 months ago. It came back for a while, but then disappeared again last month.

I have 29 pages in the sitemap and Webmaster Tools tells me 27 are indexed. There are no warnings.

When I use site: to see the indexed pages for the site, that page is not there.

When I use cache: for that page, I see a copy of the homepage.

When I use link: for that page, I see the same backlinks listed as when I use link: for the site homepage.

When I go to that page, the toolbar PageRank is the same as that of the homepage.

Checking my logs I see:
66.249.68.59 - - [11/Nov/2010:05:05:23 -0700] "GET /my-missing-page.html HTTP/1.1" 200 7186 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

I had come to the conclusion that Google just didn't want to index that page, but the cache issue makes me wonder if there isn't something else going on. Any ideas?

Thanks

[goodroi]

most websites do not have 100% of their pages indexed in google. this is generally because there are some low value pages for most websites (privacy policy, disclaimer, etc). google is more interested in indexing pages with unique content with high link popularity.

to help your internal page stay indexed you should make sure it has significant content that is unique. then make sure it has inbound links from other websites.

as your testing methods, site: search is not always accurate, link: search is even worse. toolbar pagerank also has accuracy issues.

the foolproof way to know your page is indexed by google is to look at your logfiles and see if google is sending traffic to that page. otherwise i would search for unique string of text that you add to that page.

good luck

[tedster]

You can rule out a technical issue by using the "Fetch as googlebot" tool in WebmasterTools. If the right content comes back, then the cache: issue is just a reporting bug. However, if you get back the home page, it's time to start digging.

[simplesimon]

to help your internal page stay indexed you should make sure it has significant content that is unique. then make sure it has inbound links from other websites.

Thanks goodroi. The first time the page disappeared, that's what I did - reworked the content to ensure it was appropriate and unique, and did some moderate external linking from pages that were themselves indexed by Google. For a while the page came back into the index and Webmaster tools showed these links coming into that page. Then the page disappeared again.

as your testing methods, site: search is not always accurate, link: search is even worse. toolbar pagerank also has accuracy issues.

the foolproof way to know your page is indexed by google is to look at your logfiles and see if google is sending traffic to that page. otherwise i would search for unique string of text that you add to that page.

Thanks again. I realized the inaccuracy of link: and toolbar page rank but didn't know there were problems with site: search. I also check by searching a quoted paragraph of text, and the page is not in the index.

You can rule out a technical issue by using the "Fetch as googlebot" tool in WebmasterTools. If the right content comes back, then the cache: issue is just a reporting bug. However, if you get back the home page, it's time to start digging.

Thanks Tedster. I used Fetch as Googlebot, and no problems there. So looks like a reporting bug. Guess I was grasping at straws. Is there any way to see if some kind of filter has been applied to this page? It's not spammy. The anchor text from incoming links is varied, as are keywords in the on-page elements. If it's over optimized though, it's much less than other pages on the site that are in the index.

[tedster]

Over-optimization might lower a page's rankings, but I haven't seen it cause a page to be de-indexed.

[simplesimon]

OK. Thanks for your help.