Forum Moderators: open
Thank you,
Ryan Allis
On November 15, 2003, the SERPs (Search Engine Result Pages) in Google were dramatically altered. Although Google has been known to go through a reshuffling (appropriately named a Google Dance) every 2 months or so, this 'Dance' seems to be more like a drunken Mexican salsa that its usual conservative fox-trot.
Most likely, you will already know if your web site has been affected. You may have seen a significant drop-off in traffic around Nov. 15. Three of my sites have been hit. While one could understand dropping down a few positions, since November 15, the sites that previously held these rankings are nowhere to be found in the top 10,000 rankings. Such radical repositionings have left many mom-and-pop and small businesses devastated and out of luck for the holiday season. With Google controlling approximately 85% of Internet searches, many businesses are finding a need to lay off workers or rapidly cancel inventory orders. This situation deserves a closer look.
What the Early Research is Showing
From what early research shows, it seems that Google has put into place what has been quickly termed in the industry as an 'Over Optimization Penalty' (OOP) that takes into account the incoming link text and the on-site keyword frequency. If too many sites that link to your site use link text containing a word that is repeated more than a certain number of times on your home page, that page will be assessed the penalty and either demoted to oblivion or removed entirely from the rankings. In a sense Google is penalizing sites for being optimized for the search engines--without any forewarning of a change in policy.
Here is what else we know:
- The OOP is keyword specific, not site specific. Google has selected only certain keywords to apply the OOP for.
- Certain highly competitive keywords have lost many of the listings.
How to Know if Your Site Has Been Penalized
There are a few ways to know if your site has been penalized. The first, mentioned earlier, is if you noticed a significant drop in traffic around the 15th of November you've likely been hit. Here are ways to be sure:
1. Go to google.com. Type in any search term you recall being well-ranked for. See you site logs to see which terms you received search engine traffic from. If your site is nowhere to be found it's likely been penalized.
2. Type in the search term you suspect being penalized for, followed by "-dkjsahfdsaf" (or any other similar gibberish, without the quotes). This will remove the OOP and you should see what your results should be.
3. Or, simply go to www.**** to have this automated for you. Just type in the search term and see quickly what the search engine results would be if the OOP was not in effect. This site, put up less than a week ago, has quickly gained in popularity, becoming one of the 5000 most visited web sites on the Internet in a matter of days.
The Basics of SEO Redefined. Should One De-Optimize?
Search engine optimization consultants such as myself have known for years that the basics of SEO are:
- put your target keyword or keyphrase in your title, meta-tags, and alt-tags
- put your target keyword or keyphrase in an H1 tag near the top of your page
- repeat your keyword or keyphrase 5-10 times throughout the page
- create quality content on your site and update it regularly
- use a site map (linked to from every page) that links to all of your pages
- build lots of relevant links to your site
- ensure that your target keyword or keyphrase is in the link text of your incoming links
Now, however, the best practices for keyword frequency and link text will likely trigger the Google OOP. There is surely no denying that there are many low quality sites have used link farms and spammed blog comments in order to increase their PageRank (Google's measure of site quality) and link popularity. However, a differentiation must be made from these sites and quality sites with dozens or hundreds of pages of informational well-written content that have taken the time to properly build links.
So if you have been affected, what can you do? Should one de-optimize their site, or wait it out? Should one create one site for Google and one for the 'normal engines?' Is this a case of a filter been turned on too tight that Google will fix in a matter of days or something much more?
These are all serious questions that no one seems to have answers to. At this point we recommend making the following changes to your site if, and only if, your rankings seem to have been affected:
1. Contact a few of your link partners via email. Ask them to change the link text so that the keyword you have been penalized for is not in the link text or the keyphrase is in a different order than the order you are penalized for.
2. Open up the page that has been penalized (usually your home page) and reduce the number of times that you have the keyword on your site. Keep the number under 5 times for every 100 words you have on your page.
3. If you are targeting a keyphrase (a multiple-word keyword) reduce the number of times that your page has the target keyphrase in the exact order you are targeting. Mix up the order. For example, if you are targeting "Florida web designer" change this text on your site to "web site designer in florida" and "florida-based web site design services."
It is important to note that these 'de-optimization' steps should only be taken if you know that you have been affected by the Google OOP.
Why did Google do this? There are two possible answers. First, it is possible that Google has simply made an honest (yet very poor) attempt at removing many of the low-quality web sites in their results that had little quality content and received their positions from link farms and spamdexing. The evidence and the search engine results point to another potential answer.
A second theory, which has gained credence in the past days within the industry, is that in preparation for its Initial Public Offering (possibly this Spring), Google has developed a way to increase its revenue. How? By removing many of the sites that are optimized for the search engines on major commerical search terms, thereby increasing the use of its AdWords paid search results (cost-per-click) system. Is this the case? Maybe, maybe not.
Perhaps both of these reasons came into play. Perhaps Google execs thought they could
1) improve the quality of their rankings,
2) remove many of the 'spammy' low-quality sites
3) because of #2, increase AdWords revenues and
4) because of better results and more revenue have a better chance at a successful IPO.
Sadly, for Google, this plan had a detrimental flaw.
What Google Should Do
While there are positives that have come from this OOP filter, the filter needs to be adjusted. Here is what Google should do:
1. Post a communiqué on its web site explaining in as much detail as they are able what they have done and what they are doing to fix it;
2. Reduce the weight of OOP;
3. If the OOP is indeed a static penalty that can only be removed by a human, change it to a dynamic penalty that is analyzed and assessed with each major update; and
4. Establish an appeal process through which site owners which feel they are following all rules and have quality content can have a human (or enlightened spider) review their site and remove the OOP if appropriate.
When this recent update broke on November 15, webmasters clamored in the thousands to the industry forums such as webmasterworld.com. The mis-update was quickly titled "Florida Update 2003" and the initial common wisdom was that Google had made a serious mistake that would be fixed within 3-4 days and everyone should just stay put and wait for Google to 'fix itself.' While the rankings are still dancing, this fix has yet to come. High quality sites with lots of good content that have done everything right are being severely penalized.
If Google does not act quickly, it will soon lose market share and its reputation as the provider of the best search results. With Yahoo's recent acquisition of Inktomi, Alltheweb/FAST, and Altavista, it most likely will soon renege on its deal to serve Google results and may, in the process, create the future "best search engine on the 'net." Google, for now, has gone bananas in its recent meringue, and it may soon be spoiled rotten.
Then they are pulling an AltiVista all over again. AV back in the mid 90's and it will be the end of them.
Could be. If so, that would be an example of this happening due to a general algo change, as opposed to Google doing targeted filtering. One thing that strikes me from a cursory check of that SERP is that PR seems to be much more important post-Florida to do well.
You are barking up the wrong tree. Discusion would be more productive in another direction.
You're welcome to your opinion - but we're thrashing out some ideas here. I for one have read and considered Claus's posts - in fact every post since 15 Nov.
read rumors that the new results are due to manual 'demotion' of sites based on spam reports.
IMO not a chance. It would take a huge army. Furthermore, tons of non-spammy sites are gone.
manual 'demotion' of sites based on spam reports?
There are too many sites affected - and the members of this forum would need to have been filing endless malicious spam reports on each other for the past 5 years!
Surely not - gulp ;)
Anyway, Google prides itself on negligible human intervention, they like their algos.
On page factors should play just a tiny part in the ranking of a page.
What other sites say about your site is more important.
If you really have a great site with excellent content, you should rank well for your subject matter.
Concentrating on on-page factors is a waste of resources in my opinion.
What is notable about those "foo bar" SERPs is that mostly these are unoptimized pages; many of which are quite old and haven't been modified in years. Unlikely that many have had significant changes in inbound links over a period of one month. Thus, the vast majority of this movement *can't* be explained by changes in on page, or off page, factors. It has to be Google has somehow did an algo change to cause this. Turning up the PR dial would tend to explain a lot.
[edited by: rfgdxm1 at 6:47 pm (utc) on Dec. 3, 2003]
I compared Domain names, Titles, Descriptions, Header Tags, Keyword density for each individual keyword, density for the entire phrase, number of backlinks, backlink anchore text, and PR value.
Here are some of my results: (please keep in mind I am not of results)
Pre-Florida:
Domains:
There is a pretty even split between using a branded product name and company name. 3 Sites use k1 k2 k3 as their domain name or have a second domain with k1k2k3 redirecting back to the other domain.
Title:
92% used the exact phrase within their Title
Description:an expert and I only used the top 25 results for each set
92% again used the exact keyword phrase within the description
Header Tags:
only 20% have any header tags
Keyword Density Averages:
K1: 30.82%
K2: 33.53%
K3: 34.48%
K1 K2 K3: 18.94%
Average # of Backlinks: 31.64 ranging from 6-120
Many of these sites used their company name in the anchor text on backlinks and many combined branded name + keywords. Almost all of the, with the exception of 3, are directories and link spages.
Post Florida Results:
Domains:
Very even mix of .org sites, sub domains, links pages, and both free and paid directories.
Title:
56% used the exact phrase within their Title
Description:
36% again used the exact keyword phrase within the description
Header Tags:
only 8% have any header tags
Keyword Density Averages:
K1: 32.12%
K2: 29.25%
K3: 33.99%
K1 K2 K3: 18.85%
Average # of Backlinks: 56.4 ranging from 0-565
The Majority of these sites had less than 3 backlinks to the individual pages being listed and the average number is heavily swayed by one site that had 565 backlinks (all internal from the site).
Possible penaltys for previous top 25:
combined keyword stuffing, some sites using two domains and redirecting them-one being a company or brand name and the other being k1k2k3.com or something. None of the 25 listed have hidden keywords , none have affiliate pages or obvious penalties. None seem to have broken any of the normal guidelines.
Notes:
The #1 result under the post-florida results is a site that has ABSOLUTELY nothing to do with the keyword phrase in an exact match or broad match.
Their are 3 companies that have survived this filter, and their pages are not much different than the ones that are gone. The only noticable differences are in the type of backlinks and the price structure of the products being offered.
The remaining sites have lots of internal links and links from press releases
Also, the reminaing companies have products that cost over $10k. The sites that were dropped ranged in cost from $150 to $5k.
I did this on a simple spreadsheet and would be happy to let somebody else evaluate it if they want.
[edited by: vbjaeger at 7:00 pm (utc) on Dec. 3, 2003]
<Concentrating on on-page factors is a waste of resources in my opinion.
So, Which is it Kiwi Webmaster, Onpage=(excellent content)in my opinion.
;-)
Still in the serps, spam, non-revelent sites for keywords, sites in the serps, just because of content of link from other site, other sites with nothing but search results, forums with single entry of keywords.
Looks Still broke to me. I am not yet willing to risk my ranking on other SEs, banking on G staying with bad serps.
thumpcyc
I'm suspicious of this link problem thingy
I am not, otherwise how can you explain the following:
A Certain site was banned (home page) for "blue widgets" - not my site BTW
Blue widgets is a fairly competitive keyphrase
- Title does not have "Blue widgets"
- Title has "Widgets - Blue" once
- Meta description does not have "Blue widgets"
- Meta keywords has "Blue widgets" once
- no H tags with "Blue widgets"
- body text does not have "Blue widgets"
- body text has "Widgets Blue" once
- body text has "widgets" 4 times
- no ALT tags with "Blue Widgets"
- site has a DMOZ entry
- site URL does not have "Blue widgets" or any other combination
- Majority of backlinks are in the following format "Blue widgets"
The only over-optimization, or optimization, of this site is the anchor text being "Blue widgets" for most links.
Can someone tell me anchor text has nothing to do with it, I simply don't buy it.
[edited by: defanjos at 6:58 pm (utc) on Dec. 3, 2003]
What if it isn't a penalty, but just that KWD means very little with Google today? IOW, high KWD does little good unless there are off page factors to get the page to rank well? It may also be more than just PR. PR is a sort of measure of "authority". There are other possible measures for authority. Most e-commerce sites tend to do very poorly in terms of looking like an "authority" to an algo.
What if Google decided to do this:
a) Reduce the weight, drastically, of on page factors (such as frequent appearance of keywords), but not actually punish anything.
b) Place proportionally a much greater emphasis on off page factors, such as incoming anchor text / PR
c) Reduce the predominance of index pages in the SERPs
but
d) Still allow the PR value, and anchor text from these index pages to be passed onto sub-pages.
You would get SERPs full of sub-pages, and established sites would appear to have disappeared, because they relied for their placement on the index page.
defanjos, what is the PR of this "blue widgets" site you refer to?
PR4
All links say the site is about blue widgets, but the site ITSELF barely represents blue widgets in what we all would consider normal webdesign and SEO-centric respects. This isn't a case of a keyword penalty. This is a case of a site (apparently) misrepresenting what it's about! I'm sure there are differing opinions, but just because a site mentions blue widgets a few times in the text doesn't make it a site about blue widgets.
Before I'd jump ON the anchor text penalty bandwagon, I'd lay money that the site content just doesn't match up with what the inbound links say it's supposed to be about. At least from the description you gave me. Heck, On my site I have a news article about a topic COMPLETELY at odds with my industry (simply because it was informative). But that's not what my site is about. That's not what the site ranks for... nor would I expect it to.
Other possible measures of authority, besides PR:
1) Links from directories. Particularly a link from the cat that is on topic for the keyword.
2) Intra-site themeing. This means that if the keyword is "widget", you'll do a lot better if you have lots of pages about widgets, rather than just one well-optimized page.
3) Inter-site themeing. Links from other sites count more for "widget" if those sites intra-site theming shows they are about widgets. Problem with this is that it tends to be computationally complex for a SE to do that. Google may have figured out how to do this today efficiently.
Also, say if "anchor text" = "keywords,title" penalty theory is true for a minute. That means if I don't like my competitor, I could start creating links to his site using his keywords and google will panelize him!
# SIGH!
[edited by: Herath at 8:41 pm (utc) on Dec. 3, 2003]
That's low. You stated that this page had poor on page optimization, and this was a competitive search phrase. This doesn't bode well to rank with Google, at least it seems so today.
BIG GUYS fixed budget
Small Guys (30 - 50% budget increase)
I have to pay on average 30-50% more than I used to and receive less clicks than used to I don't think it is fair.
And I have no options you have to stay in IT SUCKS....
That's low. You stated that this page had poor on page optimization, and this was a competitive search phrase. This doesn't bode well to rank with Google, at least it seems so today.
That is a valid point, but you would think that it would not do good before either. Before the update it was in the top 10.
The reason it did good before, was the anchor text, the reason it is nowhere now, is the anchor text either got it banned or is discounted totally.
Got your sticky...
Most of the sites listed post florida are directories. Both paid and free. 3 of the post-florida sites however are sites that were also previously ranked well. The directory pages and have very few backlinks if any because (I think) most people link to a home page.
The 3 sites that survived however, are not listed in all of those directories. They are listed for some like Yahoo's directory, but not in any of the smaller directories, paid directories, or dmoz. Due to the size of the companies, I dont think they went through the practice of trying to "SEO" their sites. However many of the on-page characteristics are almost identicle to the sites that were dropped.
I'd bet on discounted. For quite a while anchor text has been a very large part of the algo. The guys at the plex may have decided since this was being exploited so much by SEO types, they turned the dial way down on the importance of anchor text. Since that was the only thing that page was well optimized for, it got creamed.
The moral here is that perhaps going back to the basics of good web page design, and traditional SEO, is wise. Also, remember Google ain't the only SE out there. Relying just on anchor text ain't the way to do well on Ink and Ask Jeeves.
[edited by: rfgdxm1 at 7:31 pm (utc) on Dec. 3, 2003]
