Forum Moderators: Robert Charlton & goodroi
In further research last night I (think I) came to the conclusion that the pages that we lost were over optimizised. But in what way and where are the questions. I could only compare who is listed, and who is not.
On one of my main search terms, where last week I was numbers 2 & 3, I am now # 59. (other sections are worse, but sticking to this problem for my research)The result should be one of my deeper pages, to the actual related page of the site. But what appears here is my main index page. The page that should be here I have not found yet.
So if i am thinking i am over optimized, i want to see what others on the google results page are having success with. Guess what? They are way over optimized. But this is the difference that I can see so far. In the unlinked content on the page, they repeat the search term over and over - key phrase density probably twice as much as i have, maybe more.
But I think the difference is that where I repeat the term, it is anchor text internal links. I think I repeat the phrase in the anchor text equal to or more than I do in the content on the page itself. It does not appear spammy, it is basically the navigation links that I speak of.
So, is it possible that google is looking at the anchor text links and weighing those phrases more than what is in the content itself and considering repeating phrases within the link as spam? Could it be the anchor text density verses the density of the content itself?
Am I making any sense to anyone?
[edited by: tedster at 9:20 pm (utc) on Feb. 27, 2008]
I has ranked (up and down) within the top 10 for the last 3-4 years. Well linked by government (albeit Canadian government) and .edu sites. If that means anything at all. And a tonne of other good quality links too.
There's content, some affiliate links, etc. But man..This is scary.
Now that I've confirmed I've got this penalty I'm a lot more worried about it. It'd be nice to have SOME word from Google as to why this has been applied to our sites, so we can 'fix' them.
If they send people who use blatant spam techniques like hidden text warning emails, why not tell us 'good' white-hat webmasters if they have concerns over something we're doing?
Arrrgh.
spinnercee, I agree with you. About half of my visitors come through bookmarks, links, mail lists and word of mouth (or mouse) but the fact is that a big proportion of the other half come though Google. Google is the major way new people find my site. I can't ignore that and even though only of small portion of my site is affected by this. I want to do what I can to correct it before the problem gets worse.
madmatt69, I think there are a lot more sites affected then seems apparent. The people who lost a lot of pages noticed it first. I only noticed when I was checking the ranking for a few pages like you did. I suspect there are many, many people who are unaware to the penalty because only a few pages are affected.
Yeah I noticed the drop off for awhile, but just thought it was one of google's mood swings. The thing is that the pages that were affected on my site were the highest traffic pages, so my traffic is down to about 30% of normal. I just decided to double check a few pages and definitely can confirm this penalty.
But yeah, I can see fluctuation in the serps. But to take quality sites and so harshly banish them to the bottom of the pile makes no sense to me whatsoever.
I wish I had ideas to solutions, but I can't figure out what is causing a problem. My site is similar to the others described in this thread, well linked, several years old, trusted, etc. I use good design meaning H1's etc. I monetize the site with some affiliate links. Big deal?
I had some dupe content problems last december which were all fixed, and immediately recognized by google. And then a few weeks later banished. Now I have no clue what could be the problem.
We all know google has been slamming affiliates....
I see the penalty for us in a few phases, based on where we ranked before - it seems for us there is a -30 penalty, a -50, a -100 and a -950+
Its funny, some of the pages fall right in these new pages right in place minus the penalty.
Results per page: *10, 20, 30, 50, 100
Coincidence? error? or by design?
But having a few links to affiliate programs is certainly not an issue. If google doesn't want people monetizing their sites, they better take a good look at adsense.
There must be another culprit at work with this penalty..
<edit>
That being said, some of my pages hit do have more than a couple affiliate links on it. For example, a list of airlines. Some are straight links, some are affiliate links if the airline has one. If I'm sending them traffic anyways, why not get a commission if it leads to a sale? That's why I think it'd be silly for google to be penalizing affiliates for that.
</edit>
Just wondering if it has anything to do with the current 950 penalty.
1) There is a new aspect of the Google algo that appears to be designed to penalize spam/scraper sites.
2) If this is true we need to look at what these kinds of sites do then consider what on our missing pages is like them.
3) The penalty is unrelated to PR and possibly even to trust rank - authority site sort of factors.
4) In the case of pages that are seemingly the same in terms of linking structure, key word density and inbound links some have plunged and some still rank quite well.
5) It could be related to a recent patent that uses the density of related phrases to determine likely spam pages.
That's for starters. What else can be added?
The thing is that the pages that were affected on my site were the highest traffic pages, so my traffic is down to about 30% of normal.
Yep, the way it has been for me since this first started occuring. Nailing me on my money makers.
Exactly randle. This approach would be suicide for Google.com and would not impress any of the shareholders.
I think quite the opposite. Non-relevant results = more clicks on relevant ads.
On a side note, I have noticed my PR for pages getting penalized are 0 in the bar. Yes, I know this is not a rule to follow, however one of my pages still ranking has a PR and another one that doesn't have a PR is ranking but is definitely old enough to have PR. The two that I am really getting slammed on have no PR.
In fact, other pages of my site are now ranking for my two main target terms, but are not even remotely releveant as the ones that should be ranking because they are different products, but do contain two mentions, links in the menu, of the main target pages.
To sum it up: Pages that should be ranking on my site are getting penalized. Pages that should not rank are ranking. This must mean, that overall, my site does have value. I think I already knew that.
Enough of my incoherant ramblings for now. Bed time. It is just a mess!
5) It could be related to a recent patent that uses the density of related phrases to determine likely spam pages.
Yes, you may be onto something with the phrase filter. As a matter of fact, this is something I adjusted this evening. It is hard to adjust on a page that is showing a data table where repitition is necessary, but a data table could be a prime example of where this filter is failing.
My pages are getting cached everyday, so hopefully my changes will be reflected in results in a day or two.
Do I keep hearing affiliate links......We all know google has been slamming affiliates....
Zero affiliates with two of my sites that have suffered from operation 950.
New Patent Application - Spam Detection Based on Phrase Indexing [webmasterworld.com]
>>Number of related phrases used, with figures given as examples as to what's normal and what isn't.
>>Inlinks
>>Anchor text of inlinks
>>Outlinks
>>Anchor text of outlinks
>>And on and on it goes...
I don't think the 30+ thing and this 950+ thing are just one thing, for either one. That would be too simple
Here is what I did so far:
- Removing 4 of 10 external links on the main-page. One link went to a purely mfa-page. Putting the remaining 6 links from the footer into the text.
- Removing another external link, which was rotating to different locations randomly.
- Reducing keyword-density on the main-page of the main keyword.
- Rewriting internal navigation. I had something like <a href="page.htm" title="Widget">Widget</a> and turned it into
<a href="page.htm" title="Nice information about things">Widget</a> to avoid repetitions.
- Changing the main-title.
- I had about 30 pages with the same title/keywords/descripion like the main-page (which was a mistake of mine in my code).
I rewrited this to totaly individual texts per page.
But the pages I changed are not in the cache by now, so perhaps this is not the one which released the filter.
But otherwise, perhaps changing the main-page's title/keywords/description was enough to let google see that the main-page is different from the other pages which had the same title/keywords/descripion.
- I had about 120 pages which google took into index (not with content, but the url) like: /shop/add_item.htm?item_id=123456. /shop/ was forbidden by robots.txt, but the pages had "index, follow" set. I set a "noindex" into these pages and deleted them manually with the url-console.
- I changed nothing in the link-structure on incoming links.
So I don't know if this is going to last. I don't know as well if the "950-hell" was only one of google's mood swings or if the things I did helped me out of it (if I'm really out...). Let's see what the weekend will bring.
Now what we are seeing is that if your site is dedicated to widgets a lot of pages are at the back of the serps where google "thinks" they are related and duplicate. So you can have pages absolutely specific to something that dont rank anywhere because another page on your site either carries a "keyword anchor link" to that page or is content about another aspect of the "blue widget" and google is simply bunching the lot up as duplicate and filtering it.
That would explain a lot.
For example:
I have a site about Widgets. Widgets have been made by several different companies over the years, so they are broken down by manufacturer: Acme Widgets, General Widgets, Standard Widget Company, etc.
There are hundreds of different types of Widgets: Model 460, Type 29, Number 110, etc., and each has different colors, materials used, production methods, etc.
People interested in learning more about Widgets usually pick one brand that they like best. While similar, they are each very unique so they can't be lumped together on one page.
So, I have a page for Acme Widgets, which is broken down by model number:
Widget Model 460
It has pages for color, materials, production, etc.
Then there's a page for Standard Widgets,
Widget Type 29
Color, materials, production, etc.
So while the information on each page is unique, the theme of each page is very similar to other pages on the site, but not similar enough that it would make sense to a human to lump them all together.
If you have a car site, you aren't going to stick VWs, Cadillacs, Station Wagons, and Corvettes on the same page because it wouldn't make sense. Same for Widgets.
I think because so many of the page titles are similar, Google thinks they are the same, when in fact they are different manufacturers, different model numbers, etc. Some of the colors may be the same, just like there can be white VWs, Cadillacs, Corvettes, etc., but you wouldn't logically group them in that manner.
It comes down to Google not understanding what a site is about at all. It just sees similarities between the pages, and then says, "These are all the same, so we'll pick one and rank it...the rest go to the bottom of the pile because they are so similar."
So, if Google chooses your white Acme Widget Type 29 page to rank, all other white Acme Widget pages get penalized. So, when someone searches for white General Model 315 Widget, or white Acme Widget Type 28, both very specific searches, even though you have that exact page on your site Google doesn't show them your page, because it thinks they are the same, and your white General Model 315 Widget page is buried at #950, or whatever, because Google thinks your white Acme Widget Type 29 page is close enough.
Google needs to understand you can have a page on:
White 1973 Volkswagen
White 1973 Cadillac
White 1973 Corvette
White 1973 Widget
White 1973 Bicycle
White 1973 Tricycle
White 1973 Dishwasher
White 1973 Dishware
White 1973 Dish Towel
and while they may seem similar, they are in fact very different. 1973 could be the year, a model number, etc., and it would seem Google doesn't understand this.
[edited by: AndyA at 12:56 pm (utc) on Jan. 25, 2007]
Googles guidelines state no more than 100 links on a page!
I worked a little with madmatt and he fixed a few things on his site that I pointed out last night. It will be interesting to see his site rebound because his site was full of good content.
Google's incompetence in this is absolutely staggering, matched only by their bullheaded persistence in continuing down this drunken path.
It's mind boggling that the one thing Google has done involving this penalty is stopped pinning the penalized pages at the bottom of the site: search results, no doubt due to reading webmasterworld.
Now if only they'd actually fix their pathetic search engine....
I've been hit by the 950 penalty and I don't have more than 100 links on my homepage.
I am astounded by the quality sites that I see at the bottom of the SERPS today. Not only authority sites, but Google Books and Google Directory pages.
Nothing to do with this filter and having more than 100 does no harm, you just lose potential benefit.
Google needs to understand you can have a page on:White 1973 Volkswagen
White 1973 Cadillac
[SNIP]and while they may seem similar, they are in fact very different. 1973 could be the year, a model number, etc., and it would seem Google doesn't understand this.
Google may very well understand that, but Google may also feel that it doesn't need to clutter its search index with nearly identical pages for each model or product variation.
If that's at the root of your problem, excluding nearly-boilerplate catalog sections with robots.txt might go a long way toward solving it.
People, forget Google, Matt don't want talk about this big change, many of us are asking in his blog and nothing, I think, if he doesn’t talk, something is happening. I'm sure; this is a new AHSM model (Adwords happy shareholders model) LOL.
I'm working hard to opt. my site for Yahoo and MSN.
Best regards,
Jakomo
I have found pages on my site hit with this penalty that are not optimized, other than having descriptive title and meta descriptions. They were built a long time ago, with no thought toward optimization at all. The keyword or phrase may appear once in the body text, but that's it. So there goes the over optimization theory, at least in this case.
My main page had 70 links on it at one time, I reduced this to 50 a few weeks ago and it has had no impact on my site, one way or the other.
The pages that I feel Google might think are too similar are not boilerplate catalog type pages. They are spec pages, with unique information that has no similarity other than the description of the data on that page, i.e., wire size (1 page could be 14 gauge, another could be 12 guage fuselink), wire color (could be any color), size (inches, height, width, sometimes all, sometimes only one), material (rubber, steel, stainless, aluminum, etc.), so I think there are enough differences to warrant separate pages.
I am going to expand the content on all of these pages with unique written paragraphs, but it will only go into more depth explaining details already on the page, clarifying, etc.
When I run the pages through a similarity checker, they do not come back as a match.
[edited by: AndyA at 3:00 pm (utc) on Jan. 25, 2007]
Although I am still a newbie, my hypothesis is that their new "filter" is first (programatically) targeting specific key words that are tied into AdSense, then when these pages are found, running another script that looks at density. If you look at a lot of the spammer sites, they pick the big money Adsense terms, then bulk up their page with those terms. Unfortunatley for us, the filter is a little too inclusive and unintentionally caught some of us in its trap.
At first I was going to sit around and wait for something to happen before I made any moves. However, it seems like I am going to spend this weekend trying to minimize the number of terms on the page -- especially for H2 text, although I am going to leave my meta title/description/content tags alone (because (1)they are accurate and helpful to my users and (2) my original content news articles are set up with the title of the article in the t/d/c and H1 being the same have not been a victim of the 950). I am also going to leave all original content text alone. If I use the name Mr. Blue Widget in the biography of Mr. Blue Widget, then it does not make sense to remove it.
If it works, I'll let you know by next week if the 950 issue can be resolved with a re-index of the affected pages. (or do I need to file for re-inclusion?).
My site has been around since 2001. All original in-house developed content. No AdSense. Our interests in driving traffic are in gaining new customers. Through a mix of ongoing content additions and aggressive SEO I have seen steady growth in Google traffic over the years. We had first page results for hundreds of keywords I track. We are a sizable company in our market, but it’s in a highly fragmented services segment with thousands of competitors, so getting and staying on top requires constant work. There is no question that I push the boundaries of "search engine approved" acceptable SEO techniques.
Then... around 9:10AM (Googleplex time) on Monday the 22nd my search positions in Google disappeared across the board, along with 95% of the 700-800 referrals/day. I target a broad set of long tail phrases in my sector and about 99% of those were banished. Sometimes I would find our site in the 900 range, but for others I could not find it at all. If I was on the first page, it would usually point to some page that was "wrong". For a handful of obscure searches I would still show top position with the right page, but it was few and far between.
After seeing other reports, I figured it was my linking structure/anchor text over optimization that tripped this filter.
Then just like a switch... at 6:28PM on Wednesday the 24th, everything returned to "normal" and my traffic and overall search positions seem to have returned as before.
The only thing I changed was on that Monday night when everything vanished I did implement a change to de-optimize some of the "link stuffing" which I figured was over the top and effected every page on my site (about a 1000 pages)
So now that the traffic has returned I have the following questions:
- Did anyone else see this same pattern/timing?
- Did Google reverse course completely on this filter because it was causing some really bad results? Or did they just do a minor adjustment of the threshold of "good" vs. "bad" sites?
- Could the change I made on Monday night have been reflected that quickly (I don't think so)?
I’m happy that the light’s back on for me.
It may just be a matter of whether the pages that haven't made it back have been spidered or not. The index page is back and the articles it points to are not. So is it that they haven't been spidered or have the article pages just not made out of the dark side of that line?
I don't see this as permanent yet. I think Google is looking at results and tweaking.
One major product term, which is not even a target term of mine, is showing a site in the top 10 that is not even remotely relevant. It is completely about an unrelated occupation, not a product. In fact, the page doesn't even contain an instance of the search term anywhere. Not even in the source code. And this is a super high traffic term... easily over 2 million searches a month.
Also, I would never expect this site to do anything black hat linking related. It is a gov't site without the gov't domain, but not even related to the search term.
I would say we got caught in the middle of a test, and now they are trying correct it.
I did fix some errors in my xhtml, but nothing I consider too major. But good to fix nonetheless.
It does appear as though traffic has bounced back up this morning. I haven't checked all the DC's yet, but we'll see if it continues to hold. And the new PR7 is appearing to hold too.
Keeping my fingers crossed..Anyone else seeing changes too?
