Key individual pages dropped from index

Some 'popular' pages dropped, why?

[tigertom]

A couple of pages which are:

- More keyword-optimised that others (6.7% density)
- More linked to from external articles I wrote myself
- More linked to from other pages in my site ...

... have been dropped entirely from the index. The rest of the site seems to be indexed OK.

I only noticed this because I regularly check their rankings, as a measure of how the site is doing generally.

Any clues as to why this might be happening? Temporary glitch? Everflux? Latest penalty for SEO'd sites? Harbringer of things to come?

Sorry if this is a standard problem. It's new to me :(

[tedster]

Just to clarify, are these urls gone from the site: operator, or not showing up anywhere (even if much lower) for their related searches, or both?

[tigertom]

Yes, both. They are totally gone. 'Site:' command does not return them. They are not showing up in my keyword searches, either. Because they're gone.

Interestingly, accidental duplicates of these pages were on another site of mine, on another server. The second site should not outrank my main one; it's much less popular. Google (or any other SE) should not have been able to get at the duplicate pages. I only found them when searching on unique phrases in the de-indexed pages.

I would be very happy if that was the cause of my selective de-indexing. I've nuked the duplicates, anyway.

[tigertom]

The de-indexed pages still have PageRank, according to my toolbar. It's very odd.

Also, one mistake I may have made was to link to them in the author bio of the (off-topic) free-reprint articles I wrote.

[renomart]

I have just noticed this today. This morning my main keyword ranked 11-12 (of about 45,000,000)on google.com. This evening it has vanished.

Strangely however, the same keyword now ranks #1 (of about 1,540,000)on google.com.au where it used to be around #5-6.

[vicyankees]

i have noticed similar activity at the page level. a site: query returns 179k pages but if i search for the individual pages that no longer appear in the rankings, those pages are gone.

[tigertom]

Looks like one will have to tread _very_ carefully with SEO from now on. My site has been dropped by MSN (bots now re-visiting after a re-inclusion request), now this from Google. What next?

My opinion only: Links from quality sites, and basic optimisation seems to be the way forward, if you care about your site. No link exchanges, heavy SEO, 301 redirects, or 3,000 links from article-reprints. And no, I don't have hard data on this, just an impression from trawling this and one other forum.

If you're in the business of making throwaway domains, I don't suppose you give a damn. Google tweaks its algo? Throw up another 20 domains.

Pain in the fundament for one-site-webmasters, 'though.

Still, early days yet. Could just be another GoogleSpasm.

[steveb]

Google's lazybot crawls less and pages drop from the index all the time.

Any page with less than ten in-links could be dropped.

It's a result of the new blog-focused crawl priorities. The lazy bot loves thousands of blog comment links more than six PR4 links.

As long as duplicate content isn't involved, it's doubtful you did anything to cause the pages to be deindexed.

[renomart]

Does Google like seeing TextLinkAds on a site?

[tigertom]

My articles likely aren't to blame. I found less than 100 links to either page, in MSN.

There were two accidental duplicates of these pages, on other, much-less-popular sites of mine. They shouldn't have been accessible. I'd forgotten about them. Nuked one set, nuking the other later today. Must be careful to clean up the trash in future.

Doubt that's the cause either, unless others report same.

[netmeg]

I have sites where pages are totally gone too - as of three days ago, they don't show up in the site command, when they've been there for years before this, and they're not particularly optimized (nor duplicated anywhere else) Some of them are coming back, one page per day. Some aren't. Several client sites are all there EXCEPT their home pages. I have to think it's some kind of a glitch somewhere.

[mr27]

Tigertom says:

'My opinion only: Links from quality sites, and basic optimisation seems to be the way forward, if you care about your site. No link exchanges, heavy SEO, 301 redirects, or 3,000 links from article-reprints. And no, I don't have hard data on this, just an impression from trawling this and one other forum.'

I have a question about 'no 301 redirects'. I have heard that one should 301 redirect all non-www pages to the equivalent www pages, because otherwise Google may see the non-www page and the equivalent www page as content duplication. What do people feel about this?

Alex Reid, Cambridge UK.

[tigertom]

301 redirects refers to MSN only. People have been reporting MSN can't handle them properly.

My drastic solution:
[webmasterworld.com...]

If I was starting from scratch right now I'd consider separate sites for MSN, Yahoo and Google, focussing most of my effort on the latter.

I'm not an expert. Feel free to disregard my headless chicken behaviour.

[steveb]

If you don't have a 301 on, you're inviting Google to drop your duplicates. That is a whole other (old) issue.

[tigertom]

As SteveB says, for Google you have to 301 your 'domain.com' requests to 'www.domain.com', or suffer a penalty if Google spiders your 'duplicate' site.

Good news(?): One of the dumped pages is back. Not holding my breath; waiting to see how this pans out.

A pleasant surprise, anyway :)

[Nick0r]

I've noticed the exact same, over a month ago in fact: I made a thread ( [webmasterworld.com...] ).

I think this penalty could be due to sitewide links and/or a high percentage of similar anchor text on inbound links.

I got one sitewide link removed for one of my nuked pages and it returned shortly after.

[edited by: tedster at 9:14 pm (utc) on Jan. 13, 2007]
[edit reason] make link clickable [/edit]

[tigertom]

Thanks a lot for that, Nick0r. That thread is very helpful.

Now to figure out what could be causing that other page to 'tank'.

[netmeg]

All my pages have come back as of last night, without my doing anything. I think there's still some issues with the site command. When I did a site command on the 7 page site, 5 of the pages now show up, but still not the home page. However, the home page does show as the #1 SERP for the main search terms (that's the part that just came back - it's been gone for five or so days) So the pages must be there - they just don't show up in site.

[tigertom]

Mine are gone again, and one other page is still in the index, but nowhere when its keyword is searched on.

These were the three pages I regularly checked on Google to see how my site was doing overall, and had the most in-site linking and SEO emphasis. I'm de-emaphasising internal links to these pages. Otherwise, I'm just waiting until things settle down.

There are a lot of people in a worse situation than me in the main Google update thread, so I can't really complain.

[g1smd]

I see a 180 page site that lost 35 pages out of the index several months ago, then another 15 more recently.

I still haven't worked out any reason other than maybe not enough incoming links to the site as a whole.

[tigertom]

Found another page in 'GoogleLimbo'. There are probably more. Annoying.

[fishfinger]

The de-indexed pages still have PageRank, according to my toolbar. It's very odd.

I don't think that is odd as I read that the toolbar will estimate PR for pages it (says it) doesn't know about based on the rest of the site. If you see grey then get worried.

I lost 4 pages from a site last week - 2 were great entry pages. The site has less than 15 pages so it was quite a hit. One of them had no IBLs, the others had a few.

They are all back and ranking again since yesterday.

However on a related issue (or perhaps 2 related issues)...
I have two other sites where site: shows no pages (although allinurl: does). One site still has it's old rankings for the home page only - but for no other pages; the other has lost all rankings (we found the home page at page 80 or so of the results) for all pages.