Decreased indexed pages in google

[minii]

Hi,

Day by day google is decreasing the number of indexed pages of my website. Any specific reasons why is it happening? How should i prevent the same?

Many Thanks,

Minii

[rookiecrd1]

Over the last 5 weeks I have watched my total number of pages indexed in google drop from, 2,800,000 to 886,000. It still seems to be declining.

Anyone know of any reason this would occur?
Anyone else seeing this with their pages?

Brian

[akvelu]

I once had this happen to me and then I found out that we had replaced an inner page that was one of the entry point to a lot of links, with a new page (we organized the links in a different way) and google proceeded to drop all the pages linked through this page. Of course after the next deep crawl we able to get back most of them

[Iguana]

About three days ago my no of pages had dropped from about 80,000 to 612. It has today jumped up to 12,000. I think 12,000 may be pretty close to accurate.

[devitnow]

I'm experiening the same. 60,000 down to 500. I see heavy, heavy indexing going on. About 12,000 pages today alone so I'm hopeful it's coming back.

[ruip]

Same here 62000 to 817 severall deep crawl in last days

[headache1987]

I didn't so much notice it had dropped pages as I did that it recrawled the ENTIRE site and I do mean ALL the pages - pages that hadn't been crawled in a long time.

[walkman]

minii,
did your site get hit by Jagger? Also, what's your PR?

[walkman]

anyone else? My main site is suffering from this.
The PR dropped to 4 from 5, not a penalty I think, and my page count is dropping. Got slammed in Jagger, and Google stops by daily, sometimes 100's of times. The page total is about 100 times the real number that they have now on the index.

Manually entered pages, no dupe issues.

[Dayo_UK]

>>and Google stops by daily, sometimes 100's of times.

Are people talking about normal Googlebot or Mozilla Googlebot - there is a world of difference.

BTW - yes I have seen this too.

Normally on sites where Google can not determine the root page. I guess they are not getting crawled by normal Googlebot.

[walkman]

what's the difference Dayo?

66.249.65.199 - - [02/Dec/2005:00:38:23 -0500] "GET /mypage.html HTTP/1.1" 200 **** "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

[Dayo_UK]

Basically Mozilla Googlebot does not add pages to the index. Well very very occassionally.

Its purpose is unknown.

[BillyS]

Are people talking about normal Googlebot or Mozilla Googlebot - there is a world of difference.

Dayo -

I was going to mention this yesterday. For the last two weeks I saw very little activity other than Mozilla Googlebot. Two days into this month, Googlebot (regular one) has indexed about 1/3 of the site. I got hit hard just before J3.

Not much has been said about Mozilla Googlebot. I seem to remember GG saying it was a more advanced creature in testing. Personally, I think it's a nastly little bot that is trying to figure out if you're doing someting wrong. I've read elsewhere that it might be able to digest javascript.

It seems to be much more active (relative to the regular Googlebot) when a site is under some kind of penalty. It was very active on my site just before the regular bot appeared. I'm taking that as a sign that Google's figured out that I'm not playing any games. In other words, I'm optimistic.

[Dayo_UK]

Yes, Billy - If I was being optimistic I would agree with the theory that Mozilla Googlebot sort of maps out the site etc.

GG said this bot was in testing about 2 years ago - so by now it must have a purpose.

[walkman]

I took another look at a site of mine that is doing great in Google and has never been penalized: out of 1900+ entries for googlebot for that day, only 130 or so where from the "normal" googlebot. The rest were from Mozilla 5.0.

weird

[BillyS]

walkman - do you use a lot of javascript on that particular site? Not necessarily the stuff that comes from Google itself (Adsense...).

[walkman]

just this:
javascript:window.external.AddFavorite('http://www.mysite.com/','mysite.com')

[bether2]

Over the last 5 weeks I have watched my total number of pages indexed in google drop from, 2,800,000 to 886,000. It still seems to be declining.
Anyone know of any reason this would occur?
Anyone else seeing this with their pages?

I've been watching this happen over the past several months for one of my sites. No clue as to the reason.

[walkman]

>> I've been watching this happen over the past several months for one of my sites. No clue as to the reason.

how is that site doing in serps?

[Abigail]

it's been going on for about 2 weeks or better, little by little and then accelerating over the past few days. the other thing i noticed this morning is that my www.indexpage.com has been replaced with indexpage.com and my customized 404 page is showing up all over the place attached to dead page addresses that have been out of the index for months. anybody else seeing this kind of thing?

[walkman]

>> customized 404 page is showing up all over the place attached to dead page addresses that have been out of the index for months

I have my 404 page indexed for three pages. When tested, they return a 404 error as serve header. What's up with that Google? Most sites have customized 404s

[arubicus]

"I have my 404 page indexed for three pages. When tested, they return a 404 error as serve header. What's up with that Google? Most sites have customized 404s "

I have seen this happen also recently. Kind of funny but yet disturbing.

Moz. bot seems to act more like a link crawler in a way. Never see it index anything either. But it did hit hard recently followed by regular googlebot later that day. We have been out since allegra and slowly gaining all the pages lost back into the index. I am affraid though that with no PR (since our site disappeared for so long) we do not rank for much of anything.

[bether2]

I've been watching this happen over the past several months for one of my sites. No clue as to the reason.

how is that site doing in serps?

I lost most of my google traffic long before the disappearing pages thing started happening.

Lost most of the google traffic mid-December last year, it came back for a couple of months in the spring, then nothing until first 2 weeks in July, and nothing since then.

My pages used to rank really well in google. I don't even look for them anymore. They're somewhere down there... :)

[Lorel]

check to see if anyone has copied your site. I had a client's site copied by another site on 13 of their pages and my client's site index dried up to nothing but bogus pages. We managed to get the thief's site removed and requested reinclusion for the client's site and the Google index came back to normal in about a week.

[g1smd]

>> ...i noticed this morning is that my www.indexpage.com has been replaced with indexpage.com <<

For more than a year, there has been a post every day on this forum advising that all non-www requests, for every page of the site, are redirected to www using a 301 redirect.

Did you miss this information before?

[Abigail]

seen it, done it and that's what makes the whole thing so very strange - of course everything that's been happening seems so very strange.

[bether2]

check to see if anyone has copied your site.

Thanks, Lorel. I did have a couple of sites that copied major portions of this particular site - late last year or early this year.

Spent a lot of time and finally got them to remove the content. Haven't checked for dup content since then.

I've just decided that I'm not going to chase google traffic anymore for this site - at least not right now. Too big of an energy drain. I need to focus on my new sites.

I used to get 2500-3000 unique visitors a day from google on this site. Now I get 5-10 per day. Do I want to spend weeks or months to explore all of the 20 or so things I could do to that might bring back the google traffic? Nope. I'm moving on.

The ironical part of it is that, if I had done something that I knew might get me penalized, I could fix it and ask for reinclusion. But if you don't know why your once high-ranking pages have been dumped to the bottom of the serps, there's nothing that can be done.

[taps]

While others are wondering why Google is decreasing indexed pages I do hope that serps from my page will decrease even more.

After being punished for dupe content issue (it was a wrong line I've placed in robots.txt) I cleaned up my site and now I am waiting since the end of September to regain serps I had before.

Google decreased the number of pages indexed from ~320,000 to 155,000 now. But for two weeks now this decrease seems to be stopped.

I've seen the number of pages down at 145,000. But this was only for one day.

Normal number of serps should be somewhere around 70,000.

What is Google waiting for?

[fatpeter]

Coming back to the Mozilla googlebot question. I never realised that the mozilla one didn't add pages. So far this month I have 1400 Pages picked up by mozilla googlbot and only 8 by Googlebot/2.1 (+http://www.google.com/bot.html)

Am i looking at the problem here? or is there a chance it will add these new pages i have on my site?

[walkman]

I got 11 (real) Googlebot visits 2 days ago, 3 yesterday, and 12 so far today.
As far as Mozilla, about 600+ visits for the three days combined. I wonder why the discrepancy. This happens even on sites that have higher PR and that are doign great on Google. So good that I wish I could just exchange visitors :)