News site's posts now take a week to make it to organic SERP

[devil_dog]

I run a news website posting hundreds of news articles daily. Have been included in Google News for almost 3 years now.

Since about one month or so, our new posts are taking exactly (to the second) 7 days to show up in regular search.

The new posts are immediately pucked up in google news and frequently featured on the news cluster on organic SERPs for related keywords.

On doing a search for full title or a chunk of text from a news article (published within last 7 days), we see only scraper sites like ones who copy the title or a snippet of text and link back. sometimes even tweets show up. My site is nowhere in sight. not even with "&filter=0" . However, the link from my site is present when choosing "latest" option on Google.

One strange thing is when doing a site search with query say:-
site:example.com {story title or chink of text}
the link from my site is rightfully there.

We haven't noticed much change in Google Traffic cause older stories now are strangely performing better in organic SERP. (Google still seems to value internal links from new posts).

Anyone fill me in as to why this may be happening? has anyone faced it before?

[edited by: Robert_Charlton at 7:06 pm (utc) on Jan. 3, 2010]
[edit reason] use example.com - it can never be owned [/edit]

[Erku]

We have two news websites, they are included in the Google News and occasionally appear in the cluster as well.

We do experience the same thing described above. I don't know why scraper sites rank higher than the original news source or a uniquely written and well researched news stories. I hope Google takes some action.

[edited by: Robert_Charlton at 1:18 am (utc) on Jan. 4, 2010]

[Erku]

Anyone has any experience on this issue? Our news posts do appear in google news, but it takes days or so to appear in search organic results. Anyone knows what's going on?

[tedster]

I don't have solid knowledge, but this past month has seen a lot of anomalous behavior in Google spidering and indexing for the organic SERPs. I'm guessing it has to do with the full roll-out of Caffeine and we'll see something different and speedier later on in January.

[edited by: tedster at 7:30 am (utc) on Jan. 5, 2010]

[whitenight]

[webmasterworld.com...]

[devil_dog]

Erku: Can you check please if its only last 7 days which is missing in organic?

For me its EXACTLY last 7 days missing... i.e. 604,800 seconds...

[Erku]

Our sites have PR6 and PR4. And we have this problem. Yet I saw a bog that has PR1 and the stories submitted last night are nicely showing this morning when you search by title.

[devil_dog]

Erku: It usually takes me anywhere between 10 seconds to 5 mins to show up in organic...

dont think that PR or linkage is the issue here... it is either some sort of penalty or a glitch as tedster said...

can you confirm if its exactly 7 days for you too or its random? If its exactly 7 days then both of us suffer the same issue.

That said, I have never indulged in anything remotely Black Hat. I spend my time building content and doing a little on-site SEO...

[ecmedia]

While not talking about news site but I have also noticed that sometimes blog updates can appear in both blog and main search within seconds/minutes but some static pages can take days/weeks to appear in main search and updated pages can take months to update. I am assuming that it is just an algo change.

[Erku]

It's very hard. I even copied the title of a story published on Dec. 27 and posted in Search, it's not appearing, while it's number one in Google News.

We are not doing any SEO either. Just publishing exclusively written news stories.

[Erku]

I am starting to see some positive changes, but not there yet. Some of our stories published today started to rank in search engine organic results. But not all yet. Still confused and in the dark.

[BedSupperclub]

Did things come back to normal for you Erku?

one of my site has been cursed since February 27...
the homepage updates regularly in serps, but pages take a week or so to show up

if you found a way out could you tell us how you did it?

[devil_dog]

BedSupperclub and Erku : How many new stories do you guys publish daily? For me its 400 to 500 daily.

For me... Last weekend and Monday was very few stories, on Monday night for the first time few stories got into regular index as soon as publishing... but only 2 or 3 stories... then back to normal 7 day delay.

I have a speculation that Google fixes this delay based on frequency of new stories and domain authority.

[BedSupperclub]

10 stories per day in average here

my 1-yo site was leading in its niche so far.

Google's purgatory box is so annoying! Google's Webmaster Tool says everything is all right, but the 7-days purgatory is definitely targeting my site...


@devil_dog I see you first posted on webmasterworld on January 3rd, so 2 months later, it still isn't fixed for you.

What penalty are we victim of? It's quite depressing..

[devil_dog]

I really dont know...
Ive really thought about all possibilities i could think off... These days i see more threads on other forums about this issue... so its likely that other sites are also being hit by this. Cant find a reasonable explanation anywhere

[dazzlindonna]

devil_dog, just throwing an idea out there. If you are publishing 400+ per day, I assume lots of those get quickly relegated to archives, similar to how blog posts fall off a home page. Have you noticed that perhaps when a story hits the 7th day, it appears somewhere in particular on your site, and that particular someplace is a place that google's news bot frequents more often than other places?

[devil_dog]

nopes.. ive checked... I dont have a seven day archive or something.

Moreover it doesnt matter if the post is linked to from homepage for 10 minutes or if it was linked from all pages for days....

The bot crawls my site very very aggresively and posts are included into Google News within minutes (if not seconds).

Somehow. This problem started about the time Google launched realtime updates within regular SERP

[Rick_M]

I don't have a news site but it sounds like something Google would want to intentionally do in order to keep a news article from showing up twice on the front page. If an article is eligible for the section of related news articles, they wouldn't want it to also appear in the regular organic results.

After it is no longer eligible for recent news, it then becomes eligible for the regular organic serps.

This is pure speculation on my part but it seems logical from Google's point of view.

[devil_dog]

Rick: Excellent point... but i seem to be the only site hit by this... All other news sites seem to be showing up in regular SERP.

currently i get few backlinks per story because of the crappy design. we are in process of a full redesign to make it look more trustworthy in hope that it would incourage bloggers to link to our stories as well...

[mack]

Perhaps Google will take fresh news articles and make them search-able, but only for a short time period. Then allow a period (7 days) for articles to mature. News changes a lot, and as facts become known, news items are subject to change. I am guessing, but its possible.

Mack.

[alecs]

I was surprised to see this thread, I am currently in the process of redoing parts of my website just because of this exact issue. I am on a CMS and though google doesn't like something in the site's structure. My site is one year old and has an average of 6 articles published per day. The homepage gets cached in seconds or minutes, but articles appear in searches only after 7 days. Once in a while some of the new articles do get added into google's index but that is rare, the rest of them only after 7 days. What is also interesting is if i type site:www.example.com the new articles appear as indexed. The site has pr 3 and my guess is that is either a structural problem with the homepage or some sort of filter or sandbox effect. What do you guys think? whats frustrating is that I see many crappy sites in my niche with pr 2 for example that are getting indexed with no problem.

[edited by: tedster at 2:06 am (utc) on Mar 7, 2010]
[edit reason] switch to example.com - it can never be owned [/edit]

[JS_Harris]

devil_dog - your sites internal link structure is primarily to blame.

Sites that "pump out" the content into a less than optimized link structure can actually tell search engines that the articles are of low value because they become buried quickly with (relatively) few internal links leading to them.

I'm NOT saying your link structure isn't properly designed, I'm saying it's not the proper structure for your particular site.

[claus]

A wild guess, well... sort of:

Multiple-speed Google:

If you're a considered "trusted news provider" you get instant coffee... erhm... frequent udates.

If you're considered a "serious news provider" but haven't been fully "vetted" yet, you get the 7 day lag.

If you're not yet considered a "serious news provider" your home page may be updated but your stories may not rank.

And of course, if you're on the "superfresh" whitelist, like facebook, twitter, etc you get instant updates.

The rest of the sites out there... well, that's just web sites. Not "fresh content". So, they're subjected to the full algo, as usual. Although, with all the fresh content coming in Google is currently pretty slow for "normal" new content. So, you may also have a multiple-algo Google [webmasterworld.com]

;)

[devil_dog]

@claus : im sure its not that... cause it made no difference in Google News ranking. Only organic SERP ranking was affected. even really crappy sites were outranking me in organic.

@JS_Harris : Seems like you were right. I made modifications to the link structure... such that all stories from last 24 hours is accessible within 2 clicks from homepage.,.. and the issue was gone... i owe you a beer ;)

[claus]

@devil_dog: I'm not talking about Google News, only Google Search. Google News is an entirely different machine.

[tangor]

I'm wondering if news sites are competition for google and the other media news companies... and that delay makes sure g and friends get first bite of the news apple.

[BedSupperclub]

I'm still cursed by that Google 7 day delay...

@devil_dog: is your trick still working? or is the Google's penalty back?

[devil_dog]

it worked fine for 2 - 3 days.... then cursed again.

I think Google doesnt like me pushing too much stuff out... maybe ill try with lowering quantity... or try weeding off lower quality stuff...

<snip>

[edited by: Robert_Charlton at 4:42 pm (utc) on Mar 13, 2010]

[BedSupperclub]

<snip>

Sad to hear that this 7 day delay is bugging you again

but if that's any comfort you're not alone, i saw lot of people complaining on google's forum.

[edited by: Robert_Charlton at 4:43 pm (utc) on Mar 13, 2010]

[BedSupperclub]

One thing I noticed: if i search for my domain name with the past 24h option, only my homepage shows up, but if i click on the link "More results from mydomainname" I can see the list of the actual articles

My site is actually a blog, and the main page is showing uncut articles. So I'm going to cut my articles from now on (with something like 'continue reading')

Will tell if it works, fingers crossed!