|In and out of a penalty every 52 hours - and question about badges|
I've read several threads about the yo-yo some sites receive from google, here's my story if it can help another development team out there :
We run a large alexa top 10k website, receive over 2 million visitors a month and have over 1.9M pages indexed in google. The content is similar to digg/technorati. Starting around August 2008 we rapidly began receiving more press/links/visitors/etc..
Two weeks later, the penalty hits us again, Dec 22. Then two days later lifted, then two days later hit again. This has continued to happen up to and including today, however it's happened so many times we've started timing so that we can tell to the exact hour when it'll hit or lift. Exactly 52 hours. Literally it'll penalize for 52 hours then exactly 52 hours later it'll lift, and the cycle starts again. What's triggering it and whether we did anything to fix it is still a mystery. With 1.9M pages indexed, the effects are pretty extreme when the penalty drops and lifts.
We can see lots quality and spam sites link to us, we assume that shouldn't have anything to do with it since it's outside our control. All our outgoing links have always been NOFOLLOW to ensure we don't get caught up in link schemes.
Here's the only thing we still don't know about : image badge.
Like technorati, we offer a small little badge that people can put on their blog if they wish to track traffic on their site. Doesn't contain keywords, just a link back to our site and an small icon image. Nothing underhanded. Does this constitute a paid link even if we're not paying for it, but instead offering a service for it? We've seen too many other large well established sites doing it to assume it was the culprit. But who knows?
Anyways, i'll keep this updated with news on our experiences.
This is very interesting... and strange.
With such a big (and successful) site i'm surprised that you were given a penalty that required a reconsideration request to be lifted and that the cause was duplicate content. Btw, was the content duplicated withing your site right? Not a copy of other site's content?
What's the PR of the site? Did it change after the penalty? (not that it can be the source of the problems, but might give some hints on what's wrong)
I would be surprised if the badge harmed your site anyway. As you say it's a common practice, especially for well estabilished websites.
We have a page rank 6, we assume the duplicate content was the issue, but since google obviously doesn't reveal any information about penalties we don't know if that's the cause. Page rank hasn't changed and googlebot still spiders at the same 10 pages a second even with the penalty.
Like techmeme and technorati, the content is a sentence or two excerpt of an article, with links to the source. We were under the assumption that google had a way of dealing with same-site duplicates so we never bothered to filter out articles that were the same. Essentially we didn't know that could harm a site so never bothered NOINDEXing the dupe article. We still are not sure if that's the cause since the official blogs over at google seem to maintain same-site dupe content penalty doesn't exist. Which is why we've been looking at the image badge, (but another article by matt cutts seems to clearly state widgets+link are legitimate as long as nothing underhanded is being done)
In any case, everything on the site is 'common practice' when it comes to quoting articles, citing sources, excerpting text and following strict DMCA control. Part of our success was the fact we followed what established sites had been doing but tried to execute it better. Obviously we executed something worse.
I still think that duplicate content can't cause a penalty that needs a reconsideration request to be lifted.
About the first penalty, what was it like? Was the site deindexed completely or "only" suffered bad rankings?
And what about now? When you say you're penalized for 52 hours, what do you mean? Less traffic? Half traffic? No traffic?
Did you notice if portions of your site (pages or sections) remain "stable" even during the penalization period?
Of the 1.9M indexed pages, what percentage do you think is duplicate content?
You "noindexed" duplicate pages, did you also "nofollowed" them?
could you check in you Google Wemaster Tools googlebot's frequency graphics ?
Tell us if you have 2 peaks of the spider (in the green and blue graphics) and if they happens, as I think, BEFORE and AFTER the application of the penalty.
Because it could also be a specific spider that, checking duplicates, determines if you must be penalized.
The peaks should happen when you enter and and exit from the penalty.
Darkyl, when the first penalty hit in early december, it brought our google referrals from 80k a day down to about 1000. Then when it lifted around 6 days later, we were back at about 60k-70k referrals a day.
On Dec 22 it started to yo-yo, 60k/day for 52 hours, then 1000 for 52 hours. By doing searches for specific pages on our site google would list the page as the last result on google. Then when the penalty lifted, that same search would show us at #1 spot again.
All pages on the site are affected, only when a search including the actual domain name is used do we come us as first result. Even for specific pages on the site.
Our webmaster tools shows a steady upward line, no spikes really, just an increase in spider traffic starting in august :
dec 600k, but there is a drop right around mid december to about 300k then back to 600k what looks like a few days later
Today again, as timed, we're out of penalty. But we know it'll drop back on us in 52 hours. Does this yo-yo ever go away?
This is just the typical long-term Google strategy, quite old but it still works for them:
once again they try to force pople into Adwords, the yo-yo effect is just one example how they try to accomplish this.
SEOPTI, I seriously doubt that is the purpose. Yes, if your site traffic falls off because it gets caught in some penalty or filter, you might think of propping up your traffic with some Adwords. But even though some webmasters may react in that way, the actual cause is not going to be a Google conspiracy of some kind.
teehx1138, is your URL structure look like directory or folder, without specific file type? such as .asp, .htm,or etc....
Well, after a long night of problem solving at the keyboard and not yet on my second cup of too-late-in-the-morning coffee, I might be a bit too blunt at the moment, so please bear with me.
|the content is a sentence or two excerpt of an article, with links to the source |
This doesn't sound like a heck of a lot of content, does it? An add-on question would be: Why would Google or any other search engine want to index this "content" and not simply the source?
Again. sorry for the tone. Just can't think of a better way of asking the questions at the moment.
tedster/seopti, we've had adsense as our primary ad company since our inception so we can dispel that theory. we were on the phone with our adsense representative as soon as we had problems, she couldn't find anything wrong and just directed us to webmaster tools to see if any messages regarding the penalty.
jim, our site noindexes pages that are very brief but will let google index some of the longer quotes, however all pages display more than just the excerpt, very similar to technorati/techmeme in that there is more displayed for our visitors regarding the article that just the article. We spoke with developers at technorati to see when/what/why they noindex cited articles and followed their lead when it came to applying the same 'indexing' rational on our site.
The timeframe of the yo-yo penalty appears to be tightening. What started as a 52 hour yo-yo two weeks ago has now increased to a 17-20 hour yo-yo. We were out of penalty starting yesterday around 9:30am, google referrals return to normal levels, but had it reapply today around 3:30am, google referrals now back down to near 0.
We have many search terms specific to our site that we use to test when we go into penalty, and sure enough, we appear as result #1 when we're ok, and on page 9+ when we're in penalty. For all pages and searches. Obviously some sort of automated procedure is moving us around, the real question is whether we can do anything about it.
|The timeframe of the yo-yo penalty appears to be tightening. What started as a 52 hour yo-yo two weeks ago has now increased to a 17-20 hour yo-yo. We were out of penalty starting yesterday around 9:30am, google referrals return to normal levels, but had it reapply today around 3:30am, google referrals now back down to near 0. |
Are you able to compare the global search on separate Google TLD's to see if there are variances. They may be different , however , the point is probably the same.
Chief suspects [ without qualification ] in my mind are :
- Content - [ not enough , original or thin ]
- Badges [ who do the badge URL's point to , on topic , purpose ?]
- Internal linking [ repetitive terms Y/ N ]
- External links to your site [ maybe a pattern Y/ N ]
Although i suspect badges , they may just be part of a trigger that brings the other issues into play.
The urls to the badges simply point to the domain name, no keywords in the link, just an image with url. ie:
<a href='http://www.example.com'><img src='http://www.example.com/badge.gif'></a>
The internal linking is pretty straight forward, if the subject is Basketball (think of tags at technorati), the link will be
If it's an article,
<a href='/article/this_and_that/4000.html'>This and that</a>
External links are as usual, we get more everyday from everything from spam blogs to the new york times. Since that's out of our control we assume it can't negatively affect us, but who knows.
One of the sites that I manage just started experiencing the same Yo-Yo problem. About two weeks ago we lost nearly all of our Google traffic. Keywords that used to rank well started to appear on the last page. We figured that it was a penalty, but wasn't sure what it might be. Then, 5 days later, the site returned to its previous performance. This lasted for 4 days and then we were hit again. We are into our 4th day of this second round in the penalty box and I suspect that we will emerge from it tomorrow or Sunday. I am curious to know if this ever goes away? Or does anyone know what Google is trying to achieve with this Yo-Yo treatment?
I would try looking at the counts for the site: command when you are up an down with the yo-yo effects. My guess is that pages might be disappearing (temporarily at least) from the index when your traffic drops.
teehx1138 - We saw a site plummet and go into a Yo Yo when it added a badge from a strong PR site which was off topic . Similar to what you have described.
I can't be sure if it was the sole factor, but it's timing remains a suspicion in my mind.