Forum Moderators: Robert Charlton & goodroi
Indexing seems to have become very spotty. Some of the studies published will be indexed, but a few are not.
I checked Google Webmaster tool, and the site needs to be verified again. Last year, we had a huge duplicate title problem (running in the 5,000+ pages) that we have since resolved. The first order of business tomorrow is to make sure to verify the site.
The site never had this problem before. So my questions are:
1 - Is this something we need to start worrying about? Does this sound like penalties, manual or automatic? It's hard to explain to analysts why their papers are nowhere to be found in Google.
2 - What can we do to help Google index the pages that it missed?
3 - Is this the start of a bigger problem?
The site previously had a problem with spammy titles, which lasted about 5 days. We have since cleaned up that problem.
Again, thanks for your help
Causes can be, similiar pages being discounted, lack of internal/external link weight or positioning of link weight.
Remedies?
Often, no remedies are need, if the pages loss are comprised of non traffic producing pages.
If quality pages are dropped while others still exist, you could noindex pages that are not important.
Otherwise you could retool your internal link structure and external link structure to improve the inclusion rate of your important pages.
These are quality pages -- the studies that typically land them on CNN and Wall Street Journal. So these pages have to be visible in the SERPs
What does "retool internal link structure and external link structure" mean?
Would it do good if we remove the papers first, then make them live with a new title and new URL? If Google News will not index them, that's ok as long as they are in the main Google index.
They can't be found in Google as well as in Google News (all new papers before appears in Google News). But lately, some of newer studies are nowhere to be found in Google.
typing in the url doesn't return a listing, or you can't locate them through search terms?
Vimes.
I believe you must use sitemaps to be on G News, so thats one easy option that you must have already covered.
As you have a high PR, its not the raw number of links (although it says nothing about the relevance of those links, and it's sub-optimal to have all inbounds pointing at one page- you need deep-linking)
So, again, I would make sure the site is easy to navigate. Although I'm guessing that your 13 year old site already has this covered.
Have you changed anything recently- like trying to tell G how often things are updated for example?
When you search by exact title or URL, only the links the papers have received show up -- but not the actual papers. The site is pretty much established that other websites post the content as soon as they are made live on the Web. There are some papers where searching for it yields only the site's homepage, as the newest papers are put on the homepage.
I'm still checking what changes have been made in the past week. Even the Google Webmaster Tool has suddenly become unverified. I know that last year the site had a massive duplicate title problem (mostly a result of same titles for the HTML and PDF print versions).
Nothing published in a new folder thats already in robots.txt?
Nothing your side that is suddenly adding a meta noindex into your headers?
Try fetching the page as googlebot and checking the HTML from there. Is it as expected?
Haven't started requiring cookies or reading javascript?
