Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

WMT says pages dropping out of index, but still visible on SERPs

         

Broadway

1:26 am on Apr 7, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



In WebmasterTools, on the "Site Maps" page it says I have submitted about 200 pages but only about 150 are indexed.
This is new. Previously (last month) it showed all 200 as being indexed.
A few days ago the number was around 190, now it is down to the 150 number.

I "tested" my sitemap.xml. WMT says it tests with no errors.

Over on the "Index Status" page of WebmasterTools, it shows all 200 pages are indexed. The date of the last plot point is yesterday (so it's fairly up to date).

Two days ago, when I first noticed a hint of this problem I did searches by website directory to make sure all of my pages were shown in the Google SERPs.
site:example.com/directory1
Site:example.com/dircetory2
and so on.
I found all 200 pages listed on the SERP's page.

Today, when I saw the 150 pages indexed number, I searched Google for each individual page:
site:example.com/directory1/page1.html
and so on.
And Google showed a single SERPS listing for each one, just as it should.
I used the sitemap.xml file as my source of URL's. There is no question I searched for the exact pages I had submitted.

So, both of my tests (a few days apart) show all 200 pages in the Index, as does WMT's "Index Status" page.
But the WMT "Site Maps" page that shows that recently this number has deteriorated to only 150 indexed.

Is this discrepancy between parts of WMT common?
Have I done my testing right?
Is there other testing I could do to figure this discrepancy out?

lucy24

6:04 am on Apr 7, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Is this discrepancy between parts of WMT common?

Yes.

Robert Charlton

6:38 am on Apr 7, 2015 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Google's index is a huge system of NoSQL databases, so large that there are databases just for controlling the sequence of operations.

Reporting necessarily comes as last in the sequence, and the lowest priority in terms of resources, and I would never pay much attention to short-lived discrepancies between WMT and the serps.

Elsa

8:49 am on Apr 7, 2015 (gmt 0)

10+ Year Member



I don't understand why the index states says that I have more than 1000 pages are indexted. but I don't have so many pages. The sitemap says I have 138 pages indexted in 140 pages. however I have more than 2 pages that are not indexed..Dose that means there is a problem with my website? <snipped poster's site>

[edited by: Robert_Charlton at 9:15 am (utc) on Apr 7, 2015]
[edit reason] Sorry, no public site reviews. Read forum Charter. [/edit]

netmeg

12:02 pm on Apr 7, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I don't understand why the index states says that I have more than 1000 pages are indexted. but I don't have so many pages.


Look for extra pages with URL parameters, tags, archive pages, session IDs, etc. Are you using WordPress or some other CMS? If not set up right, they create a lot of duplicate content pages.

lucy24

6:41 pm on Apr 7, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



URL parameters, tags, archive pages, session IDs, etc.

Last time I looked, "links to your site" on my wmt includes seventeen from WebmasterWorld. That's pretty impressive, since the only place you're allowed even to mention your own site is in your profile: a single page.

Even this pales by comparison to one blog article which I know for a fact mentioned one page on my personal site several years ago. (It caught my attention at the time because it was a brand-new page and I was watching it closely.) Thanks to a combination of auto-pagination and "lang=" parameters*, it is now up to 31. Sooner or later it will hit triple digits. Incidentally, kudos to Google for listing this group in wmt under the site the page is now on, which is not the same as the site it started out on. "Via this intermediate link" is, fortunately, not always a figment of the search engine's imagination.


* I have to assume this is a WP navigation thing-- the kind people have occasionally posted questions about-- because I really doubt the blog itself comes in multiple versions. If I wanted to further inflate my link count I'd visit the blog myself while setting the brower to assorted obscure languages ;)

netmeg

8:09 pm on Apr 7, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That doesn't sound native to WP, Lucy, but it might be some plugin or plugins. WP does create a lot of dupe content, but not the lang= stuff I don't think.

Broadway

4:28 am on Apr 8, 2015 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I really appreciate the feed back. Today it shows yet another drop on the "Site Maps" page. The "Indexed Status" page still shows the right number but it hasn't updated for yesterday yet. I can't be checking for 200 URL's every day, so I'll trust this will resolve.
I can say my traffic seems normal enough. Certainly there's no sign that 25% of my pages have dropped out of search.