3 days downtime results in deindexing 90% pages?

[mr1337]

Hello,

Im am posting this, to determine the possible effects of server downtime, and the effects it can cause to the number of indexed pages in google.

What happened is to me is: 1 server had some downtime, total of 3 days with up to 11% downtime per day within the period of 1 weeks.

Summary downtime:
Data from webmaster tools:
day 1: 3% downtime - 12 errors
day 4: 9% downtime - 25 errors
day 11: 11% downtime - 98 errors

The Result: 2 days after the 3rd day with downtime, approx 95% of the pages are deindexed.
By using the "site:" command i have the following data:

site 1: 4900 pages reported on search page 1 -> actually indexed 132
site 2: 700 pages reported on search page 1 -> acually indexed 92
site 3: 21000 pages indexed on search page 1 -> actually indexed 143

8 days have passed with 0 crawl errors, since the last downtime.

For the largest site, the "traffic" details in webmaster report:
10-6 : 280 pageviews {before many pages were dropped}
11-6 : 8 pageviews
12-6 : 10 pageviews
13-6 : 78 pageviews


So, the main question would be:
Can a few hours hours of downtime, spread over multiple days, within the timespan of 12 days week result in deindexing of approx 95% of the pages per domain hosted on the server.

Secundary Question would be:
Is it more likely that this is the result of a penalty.


Regards,
Arjan

[mr1337]

In the webmastertools, the server connectivity status went to "ok", and no later than 1 hour later the pages returned completely.

yay!

[aristotle]

Normally that shouldn't have caused so much de-indexing. Was there also a corresponding big drop in Google traffic?

Anyway, I'm glad that the pages have returned.

[mr1337]

Yes there was a relative big drop in traffic, for a couple of days for all sites on the server, clicks went back to < 5 in webmaster tools, a reduction of approx. 90% (the site(s) are not heavily visited )

The visible result was, that there where no pages in the search result pagination after page 3, (i had set results to 100 per page)

The actual amount of pages indexed was reduced to a little less than 300.

The estimated result count was still the old count (a little over 20.000 according to google), but when you clicked to the last page of the search results (page 3 in my case), the estimated result count showed 270

The sites on the server don't interlink with each other, so i could'nt imagine a penalty on all sites.

But now i see that the situation had reverted back :S

Actually when i previously said, the pentalty is removed, it was more a guess, since when i clicked on result page 2, it showed all results.

But when clicking on page 1 or 3, the "max 300 indexed penalty" was still active. I assumed that the penalty was gooing away.

So i am keeping my fingers crossed that the reverting again is a data-centre diffrence or whatever, and all will be ok tommorrow.

The time for the "penalty" to go in effect was 3 days, so i can only assume that if the penalty to go away, will be 3 days also.

Excuse me for my bad english.

[mr1337]

I have just disovered, that when i search when i am logged in to my google account - the penalty is active.

When i logout of my google acount and do the same search, all pages are there, no sign of a penalty.

Very misterious.

[aristotle]

Sometimes these various methods of checking can give misleading results. It can depend on being signed in or out, your geographic location, etc. What matters most is how much traffic Google is sending you. That's what you should be watching.

[content32]

what google should do is rather if they detec a website is 404 hide it from the serps until its online no deindex or devalue... just hidden and it reappaears again when sites back up...... obviously this would be google being helpful and that would never happen.

[mr1337]

@aristotle : traffic is down like almost 100%

@content32 : the server never returned 404 codes during the downtimes.
Webmaster tools - "connectivity problem, google cannot access your site"

More observations:

-Google is frequently crawling the site

-It looks like the titles and descriptions have reverted to a previous crawl, because many titles don't match with the current title

-The site was about 1.5 month old.

-No reports about any penalties in Webmaster tools

[mr1337]

Update.

No sign of recovery, 6 days have passed since the initial traffic drop.

I read that nobody actually fully recovered of such a "penalty".

First thing that comes to my mind is:

-Because of the downtime, the domain authority is set to 0, so all links currently don't count, and you become more sensitive to penalty's.

-Because of the authority drop, your site is not completely indexed.

I have no idea why the "estimated results" does not reflect the correct amount of pages.

The "indexed pages" count went from 20.000 to approx 27.000 pages today

The crawler is still crawling as if there is no tomorrow.

Still only approx 300 pages indexed.

[mr1337]

If i leave my de-indexing problem om the side, and check out some other sites with the site: command, i have the following results:

With 100 results per page (and clicking: "show duplicates" on the last page)

site:google.com ( about 2.830.000.000 results) max navigation page 7

site:google.nl ( about 5.160.000 results ) max navigation page 8

site:webmasterworld.com ( about 275.000 results) max navigation page 7

Are these results the same for everybody?

Since when did google started showing less then 1000 results for the site: command?

It's obvious that my problem has nothing to do with the above, since the "about xxx.xxx results"-number does not change for the above sites, but i think it's interesting anyway.

[bhukkel]

yes same for me.

There are some changes in the SERPs with the domain clustering update.

[aakk9999]

With 100 results per page (and clicking: "show duplicates" on the last page)

site:google.com ( about 2.830.000.000 results) max navigation page 7

site:google.nl ( about 5.160.000 results ) max navigation page 8

site:webmasterworld.com ( about 275.000 results) max navigation page 7

I think this was always like this, at lest it was for a very long time. The number of results are estimate. But if you try to page through results, I believe you cannot see more than few hundred of SERPs listing at the best.

[lucy24]

The ceiling has always been 1000; you can't search for "cats", change the address bar manually and view result no. 500,000.

The question is what to think when you scroll through your allotted 1000 results until you get to "701-785", and that's all there is. In the case of WebmasterWorld, it can't be because several hundred pages have been deleted since the last time they crawled the site (five minutes ago).