Pages in Google main index are decreasing since Middle of August

[Cyclob]

Hi...

I use site:www.mysite.com/* command to check pages in Google main index and the number is decreasing since the middle of August.

I tried to check many things such as double check if they have nofollow,noindex, check on robots.txt to see if it blocks google from crawling the site but everything was set correctly.

Anyone experiencing the same situation as mine?

Thanks in advance for every comment.

[Winooski]

I have three sites for which I track the Google [site:example.com] query on a bi-weekly basis.

Since May, two have gone up substantially (+68%, +58%), and one has gone down a bit (-13%), but that one's been more of a slow decline since May. I'm definitely not seeing any sharp drop in August.

Could the new drop some folks are seeing be some new duplicate content filtering that's being more aggressive than in the past? Just thinking aloud...

[tedster]

After a quick study, I think I'm seeing a drop of pages that are both low PR and unchanged in a long time.

[Gemini23]

Just to add yet another site that has bombed since 25th August... website had about 30+ pages in top 30 results on Google... homepage was no. 1 on good search term.. now no. 25 and ALL other pages with search terms nowhere to be seen... traffic of 900+ per day now down to less than a hundred.. nearly all pages had Google Adsense on ... so that has bombed... good content... good links.. good service for customers... thanks Google... but I ask... why is it onyl some websites that have dropped... most of the other sites that are in top ten for search terms are still there... have Google changed their Algorythm? etc?

[SEOPTI]

They haven't changed anything, crawling is huge, they just don't index because they seem to need their server resources for:

- exporting visible PR and
- maps update and
- making the suppl. index searchable

[nomis5]

The posts here are from those who are down in the SERPS, why so few posts shouting I'm up, I'm up? For info, I'm neither up nor down but my new pages are just not appearing in the SERPS. Someone must be going up the SERPS if many are going down?

This has got to be a major problem for Google I think. How long before it gets to Joe Public and they decide that Yahoo or MSN are a better Search Engine? Does anyone else feel that this roller coasting in the SERPS has gone on a tad too long with no apparent reasons or results?

[Bones]

After a quick study, I think I'm seeing a drop of pages that are both low PR and unchanged in a long time.

It does seem like the PR importance knob has been twiddled.

I'm not sure that unchanged-ness is a big factor though. I've added meat to a lot of pages recently, and they've pretty much all seen a drop to (what appears to be) supplemental.

The other thing that got me thinking earlier was that MC mentioned about trying to keep under 100 links per page, so perhaps any links after that amount aren't receiving juice. If they don't have any external backlinks, is that what's causing them to drop to supplemental? So it might be more of an internal PR flow/link structure issue.

Are people with high PR homepages experiencing them same decreases?

[SEOPTI]

nomis5, this is not about up and down, this is about crawling and indexing patterns. If a site is up or down, that's another thread.

[nomis5]

seopti, the thread is about the down in the SERPS. My question is, OK there are are a lot of downs. But how come no posts of the ups? I'm turning the original post on its head in attempt to get an answer about why the downs. Why is no one posting that they are going up? For every down there HAS to be an up. There absolutely has to be a pile of sites going up the serps if this thread only contains posts about the downs. Come on you "up the SERPS", let us know why and how you are doing it! That's what the original poster wants to hear about.

[TheSeoDude]

Maybe because those that have gone up and replaced the white-hats around here wear a different hat color and inhabit other forums?

I think the 'evil' labeled sites have gone up.

Hope to see this change soon.

[followgreg]

I was about to post a few words here but finally it's not even worth it.
This concludes 2 months of cheap websites, no content getting to the top for no reason a human being could see.
But the last few days are really above and beyond anything else, yes "evil" is up, cheap is up, scrappers get a quantum leap into being authoritative....just wrong.
The only excuse I could find is that Google is working out some technical stuff.

Tedster >> I have examples of pages updated regularly and did drop and pages that were absolutely never updated for almost 2 years that are doing well reaching pos. 1.

[tedster]

the thread is about the down in the SERPS

Actually (to re-focus this thread a bit) the opening topic was about a decreased in the NUMBER OF PAGES in the main index. That's not the same as a decline in rankings - we have a running thread about August SERP Changes [webmasterworld.com].

I've now looked at several of the sites I work with, and the number of main index pages does seem to be down, generally, though not as severely as the first domain I checked. Cleaning house? Making room for some further moves?

I'm also thinking about the fact that Google does know how many times a url even gets an impression on a search. If the searches that bring up an actual impression for a url are relatively rare, then maybe it deserves to be sent away from the main index. Even off there in Siberia, the most important keywords can still tap the url for a search result - but no longer everything on the page.

[TheSeoDude]

I'll refocus this thread with a little shade of good news.
Some pages lost for the past 3 weeks have returned into main index.

Few returned, but a ray of hope is lighting the sky again. And they have new cache dates too. Also pages created 2 days ago are now in index.

My homepage even if crawled daily is still 9th august cache but maybe we'll see something nice on Monday.

Enjoy the weekend.

{edit} : Strong fluctuations in page counts between datacenters; and supplementals seem to decrease.

[edited by: TheSeoDude at 12:22 am (utc) on Aug. 31, 2007]