PPC_Chris - 8:44 pm on May 25, 2011 (gmt 0)
Does anyone know of the best way to get a lot of pages removed from Google's Index. Pre-Panda, my site had about 100K pages indexed. Then we got hit by Panda 2 on April 11. We decided to get rid of the vast majority of these (say 97K of the 100K) that Google might consider thin pages.
So even though these pages accounted for a very low percentage of our traffic and an even lower percentage of our revenue, they did make up 97% of total pages in our site. So, on April 25, we used 301 redirects to point all of these pages to the appropriate higher level pages in our site and updated our sitemap in GWT to show only the 3K pages still live.
After reading some comments from the Google support forums and on these boards, earlier this week we decided to 404/410 all of the thin content pages, as this would be a better signal to Google that these pages have been permanently deleted. The problem is that Google still has about half of the pages that we have removed in its index.
If my thinking is correct then Google's algorithm could still consider the vast majority of the pages in our site to be of low quality... 50K 404 pages still indexed compared to only 3K of pages Google theoretically considers to be high quality pages.
According to GWT, Googlebot is crawling about 6-10K pages/day, but only de-indexing about 1-2K/day. I worry that if and when Google decides to re-evaluate sites that were Pandalized, it is going to appear as though the majority of our pages are still thin.
So, bottom line is does anyone know of any way for me to speed up the de-indexing of these 50K pages? The only crazy idea I can think of is to resubmit our sitemap with the full 100K and hope that this speeds up Google's crawling of these pages. But it seems insane to submit a sitemap where 97% of the URLs are 404s, especially when it seems we are already in the doghouse with Google. I can't imagine I would do this.
Does anyone have any ideas that can help us get rid of these pages from the index at a faster rate?