Forum Moderators: open
Thank you,
Ryan Allis
On November 15, 2003, the SERPs (Search Engine Result Pages) in Google were dramatically altered. Although Google has been known to go through a reshuffling (appropriately named a Google Dance) every 2 months or so, this 'Dance' seems to be more like a drunken Mexican salsa that its usual conservative fox-trot.
Most likely, you will already know if your web site has been affected. You may have seen a significant drop-off in traffic around Nov. 15. Three of my sites have been hit. While one could understand dropping down a few positions, since November 15, the sites that previously held these rankings are nowhere to be found in the top 10,000 rankings. Such radical repositionings have left many mom-and-pop and small businesses devastated and out of luck for the holiday season. With Google controlling approximately 85% of Internet searches, many businesses are finding a need to lay off workers or rapidly cancel inventory orders. This situation deserves a closer look.
What the Early Research is Showing
From what early research shows, it seems that Google has put into place what has been quickly termed in the industry as an 'Over Optimization Penalty' (OOP) that takes into account the incoming link text and the on-site keyword frequency. If too many sites that link to your site use link text containing a word that is repeated more than a certain number of times on your home page, that page will be assessed the penalty and either demoted to oblivion or removed entirely from the rankings. In a sense Google is penalizing sites for being optimized for the search engines--without any forewarning of a change in policy.
Here is what else we know:
- The OOP is keyword specific, not site specific. Google has selected only certain keywords to apply the OOP for.
- Certain highly competitive keywords have lost many of the listings.
How to Know if Your Site Has Been Penalized
There are a few ways to know if your site has been penalized. The first, mentioned earlier, is if you noticed a significant drop in traffic around the 15th of November you've likely been hit. Here are ways to be sure:
1. Go to google.com. Type in any search term you recall being well-ranked for. See you site logs to see which terms you received search engine traffic from. If your site is nowhere to be found it's likely been penalized.
2. Type in the search term you suspect being penalized for, followed by "-dkjsahfdsaf" (or any other similar gibberish, without the quotes). This will remove the OOP and you should see what your results should be.
3. Or, simply go to www.**** to have this automated for you. Just type in the search term and see quickly what the search engine results would be if the OOP was not in effect. This site, put up less than a week ago, has quickly gained in popularity, becoming one of the 5000 most visited web sites on the Internet in a matter of days.
The Basics of SEO Redefined. Should One De-Optimize?
Search engine optimization consultants such as myself have known for years that the basics of SEO are:
- put your target keyword or keyphrase in your title, meta-tags, and alt-tags
- put your target keyword or keyphrase in an H1 tag near the top of your page
- repeat your keyword or keyphrase 5-10 times throughout the page
- create quality content on your site and update it regularly
- use a site map (linked to from every page) that links to all of your pages
- build lots of relevant links to your site
- ensure that your target keyword or keyphrase is in the link text of your incoming links
Now, however, the best practices for keyword frequency and link text will likely trigger the Google OOP. There is surely no denying that there are many low quality sites have used link farms and spammed blog comments in order to increase their PageRank (Google's measure of site quality) and link popularity. However, a differentiation must be made from these sites and quality sites with dozens or hundreds of pages of informational well-written content that have taken the time to properly build links.
So if you have been affected, what can you do? Should one de-optimize their site, or wait it out? Should one create one site for Google and one for the 'normal engines?' Is this a case of a filter been turned on too tight that Google will fix in a matter of days or something much more?
These are all serious questions that no one seems to have answers to. At this point we recommend making the following changes to your site if, and only if, your rankings seem to have been affected:
1. Contact a few of your link partners via email. Ask them to change the link text so that the keyword you have been penalized for is not in the link text or the keyphrase is in a different order than the order you are penalized for.
2. Open up the page that has been penalized (usually your home page) and reduce the number of times that you have the keyword on your site. Keep the number under 5 times for every 100 words you have on your page.
3. If you are targeting a keyphrase (a multiple-word keyword) reduce the number of times that your page has the target keyphrase in the exact order you are targeting. Mix up the order. For example, if you are targeting "Florida web designer" change this text on your site to "web site designer in florida" and "florida-based web site design services."
It is important to note that these 'de-optimization' steps should only be taken if you know that you have been affected by the Google OOP.
Why did Google do this? There are two possible answers. First, it is possible that Google has simply made an honest (yet very poor) attempt at removing many of the low-quality web sites in their results that had little quality content and received their positions from link farms and spamdexing. The evidence and the search engine results point to another potential answer.
A second theory, which has gained credence in the past days within the industry, is that in preparation for its Initial Public Offering (possibly this Spring), Google has developed a way to increase its revenue. How? By removing many of the sites that are optimized for the search engines on major commerical search terms, thereby increasing the use of its AdWords paid search results (cost-per-click) system. Is this the case? Maybe, maybe not.
Perhaps both of these reasons came into play. Perhaps Google execs thought they could
1) improve the quality of their rankings,
2) remove many of the 'spammy' low-quality sites
3) because of #2, increase AdWords revenues and
4) because of better results and more revenue have a better chance at a successful IPO.
Sadly, for Google, this plan had a detrimental flaw.
What Google Should Do
While there are positives that have come from this OOP filter, the filter needs to be adjusted. Here is what Google should do:
1. Post a communiqué on its web site explaining in as much detail as they are able what they have done and what they are doing to fix it;
2. Reduce the weight of OOP;
3. If the OOP is indeed a static penalty that can only be removed by a human, change it to a dynamic penalty that is analyzed and assessed with each major update; and
4. Establish an appeal process through which site owners which feel they are following all rules and have quality content can have a human (or enlightened spider) review their site and remove the OOP if appropriate.
When this recent update broke on November 15, webmasters clamored in the thousands to the industry forums such as webmasterworld.com. The mis-update was quickly titled "Florida Update 2003" and the initial common wisdom was that Google had made a serious mistake that would be fixed within 3-4 days and everyone should just stay put and wait for Google to 'fix itself.' While the rankings are still dancing, this fix has yet to come. High quality sites with lots of good content that have done everything right are being severely penalized.
If Google does not act quickly, it will soon lose market share and its reputation as the provider of the best search results. With Yahoo's recent acquisition of Inktomi, Alltheweb/FAST, and Altavista, it most likely will soon renege on its deal to serve Google results and may, in the process, create the future "best search engine on the 'net." Google, for now, has gone bananas in its recent meringue, and it may soon be spoiled rotten.
I would like to avoid the arguments over quality and just get down to some specific analysis that looks at what changes have occurred.
The main things I have noticed within the real estate industry is that a high PR has little impact on serps, while an edge has been given to directory types. Perhaps stemming or a bump given to outbound links. What I am not seeing is any seo, anchor or site design issues - it appears to be mostly onpage factors.
1. If the search term is a money-term (whether it comes from Adwords or not), apply the filter.
2. If filter is applied, show the Froogle links.
Theoretically, this would make some sense, if they were trying to separate informational searches from buy product searches.
HOWEVER...here's where it gets completely botched up. There are lots of money-terms that are filtered, but do not show the Froogle links. Why? Because those money-terms do not correspond to buying products. Instead it may relate to services being sold, for example. There is no actual product to pack into a box and send to the post office. So, these pages disappear from the google listings, and are not available on the froogle listings.
Effect: Entire industries wiped out
Google really needs to rethink this whole thing.
How is Pyrotechnics > Fireworks related to buying cakes. The word cake did not appear even a single time in the above Google Directory.
Simple Google should improve.
Powdork, you are right on target there, i've been trying to say it quite a few times in various threads, but perhaps i've not been clear enough.
Google is indeed very sensitive about the URL these days, even right down to page names. It's a duplicate issue. Say (just a randomly chosen name): "SpecialPackages.html" and "specialpackages.html" need not be the same page, and www.example.org and www.example.org/index.html need not either (two additional possibilitiesw with and without www). It's the same thing when you've got incoming links that have the URL spelled differently ("/index.htm" vs "/"), and such.
Text case and subdomains do matter. See post #84 here [webmasterworld.com]
From the same thread, msg #13 [webmasterworld.com] is sound advice regarding www - the same goes for separate versions of individual pages. Oh, and it's also a very good idea to check your server headers [searchengineworld.com] to see what really happens when a request hits your server.
Some particular servers are ambiguous as to which exact location they think they are on... can't really explain this further, but if you suspect some issues, compare server status codes and anything with "Location" in it with competitors that are not dropped.
I do honestly believe that these issues are to blame for a lot of the recent dropouts of index pages etc. Not all, perhaps not even most of them, but definitely a lot of them. I'm tempted to say that if your site was only partially hit, then this would probably be the only problem you've got - still, i know there might be other issues, so i'm not saying that. Some people may have entirely different problems, so i'm not going to promise anyone an easy fix.
And this is neither blackhat/spam/nor commercial sites - it's all across the board. (Even for .org sites ; ) Now, GoogleGuy have mentioned that they try to indentify the "right" URL's, and that seems to be true, afaik, but still, there's no harm done in helping them, or at least making sure you're not making it difficult for the Gbot to find it's way.
/claus
What has this category to do with buying books.
Stumps me :(?
Hahaha. I think it is very funny.
Why are such irrelevant categories returned.
The Chicken Foot Dominoes Game includes everything for hours of Chicken Foot Domino fun! Cluck! Press the center domino and hear the chickens cluck!
Not a joke! Don't you guys know anything ;)
Chicken and Dominoes are very closely related :)
p.s. and the reason why Buy Rubber brings up the Pets category is due to the existence of rubber toys for dogs.
This broad matching (or whatever they have done) certainly appears to be er, imaginative in terms of relevance.
hahahaha, no wonder violence is on rise these days, blame it on Google.
I do honestly believe that these issues are to blame for a lot of the recent dropouts of index pages etc. Not all, perhaps not even most of them, but definitely a lot of them.Maybe we're talking about two sides of the same coin but what I am referring to is sites picked as the authority partly based on the number of pages they have. So rather than saying some are dropped because Google is finding duplicate content, I am suggesting that sites are being given a boost because Google is not recognizing duplicate pages as such. Smaller sites with technical glitches (on their end or Google's) may get 20 pages added to their 60 while larger sites may get credited for thousands of pages.
With Froogle now appearing in some SERPs, I think it is a simple case of Google using the Froogle technology to identify commercial sites, and dropping them in the listings in anticipation of the results appearing in Froogle.
#328 in [webmasterworld.com...]
This was the start of a new thread but it got incorporated into this one but was drowned a couple of pages back from the current discussion.
I'm just highlighting a possible way that Google could turn page rank on its head and use a separately stored index of BadRank or negative page rank to act as a simple but elegant spam filter.
If this was in use it would explain all of the anomolies that we are seeing.
Please take a couple of minutes and read the post then tell me I'm going completely crackers.
Best wishes all
Sid
I searched Google for keywords "buy books" and the Google directory category returned on top of search results was
Category: Business > Major Companies > Publicly Traded
What has this category to do with buying books.
In the serps, check out the 6th result grouping : MSN. The serps simply shows the url. What value is that page to a surfer if he does not know what's he going to get?
that one makes some sense at least
haha!
PS: the only modif. I did to the home page was to remove the H2 tag, I noticed that G has December 5th as the last index date and by going to view the source of the (cached)changed home page I see that the H2 tag is in fact no longer there.
Hope this info helps.
ref: going round in circles.
No we're not - the simplest solution to this mess has been already proposed in the past few hours, by myself and others - but who cares about precedence. There *is* a filter in place, and it is a commercial one. But it is *not* sophisticated: sites identified as been commercial (to date) are being earmarked for display in Froogle.
You might dislike its simplicity, and its lack of complication - but you can't deny its elegance, and the fact that it fits all of the facts seen to date.
Yes. Commercial and non-commercial serps, and a thousand plus FU posts here at WebmasterWorld (belive me, i've read all of them, some even twice or more). Added: i had a major project coming up around Nov 15, i had to postpone it due to this - had to know if i should recommend "go" or "cancel"; still not sure.
Still, (i'd like to emphasize this) i don't think i've got the solution to it all. After all, if you've got a bunch of PhD's working for six months, you should really expect some level of sophistication. Also, people outside Google are not supposed to be able to figure this out in the first place.
>> Are the theories about money word filters, stemming, theming and such topic specific?
I'd turn it around and say that within the English-language part of the web, there are some topics or keywords that haven't been "matched" yet. You might want to read this post: Msg. #16 in "Google and Stemming" [webmasterworld.com]
It's still not "money word filters", just "terms" like in "any old term, including moneyterms", see post #6 of this thread [webmasterworld.com] for examples.
>> Is the directory syndrome a broad or narrowly found issue?
As nileshkurhade has pointed out with examples in recent posts of this thread, the directory matches to certain keywords and phrases are not the best. The directory is (at some places) a mess, as cats have been moved and so on.
I'm personally pretty sure there is some kind of connection between SERPS and CATS, but i'm not sure if it's used for "theming", for "selecting authority sites" or for something completely different. (now, who can find that one single post in the Update Florida thread part 1-4 for me, where someone claims he has seen six directory cats on one set of serps and can't reproduce it?)
>> Maybe we're talking about two sides of the same coin
Sounds like it. Sounds odd, i guess we have to dig deeper to find the answer. I'm speaking about duplicate content causing problems, while you seem to have found it to benefit some sites. Are you sure these are 100% duplicate pages (identical) and not just 90-something% (very similar, almost identical)?
>> BadRank or negative page rank
Read the paper, i don't think this is an update florida issue, although the theory is interesting. What was the date of that article? I only saw the year 2002?
>> check out the 6th result grouping : MSN. The serps simply shows the url
Just like before Florida, backlinks rule. Things like this was the reason for the edit in my last post.
>> So something is in fact happening,
Has been happening for at least four days now. Don't just watch one set of keywords ;)
/claus
[edited by: claus at 11:42 pm (utc) on Dec. 6, 2003]
Are you sure these are 100% duplicate pages (identical) and not just 90-something% (very similar, almost identical)?Turns out, no.
Poorly worded on my part. What I meant to ask was "Is the preponderance of directory type listings in the serps a broad or narrowly found issue"? Is it happening in other industries to the same extent it is found in real estate related searches?
It's plainly obvious there is no "filter". Sites are being ranked differently, not filtered.
Here's the deal... If sites were being filtered, then the sites *not* being filtered would would be unaffacted, right? That's the point of the word "not", right?
But of course that is not what is happening. One site might be ranked #3 and #9 under the old and new algorithms, and another site might be ranked #8 and #6 under the old and new algorithm. If there was a "filter" the first site would always be above the second.
Then what about the sites that are ranked #30 the old way, and jump up to #2, ahead of both of the above sites?
Using the word "filter" is betraying a lack of understanding of what is going on here.
(Duplicate content may in a sense be being filtered and removed, but that is not what is being talked about here.)
This is a total blind alley. I've been suffering from duplicate pages for months (non-www v www), but still ranked highly pre-Florida. It may have hit my position slightly pre-Florida, but it is nonsensical that this might explain a drop from top 5 to below 500 position in the SERPs!
I now have a 301 in place, but it is like chasing a black cat in a dark coal cellar to look at duplication as a cause for the massive change in page positioning.
Its time for some straight and simple thinking!
Froogle
That's it!
This 'filter', if any, is applied to specific searches. Once triggered, then a different algo kicks in.
Why couldn't google test stemming, or something tied to ctr or anything else for that matter in just certain segments first?
<edited by kirby for typos>
Definitely broad - not only directories, but also infopage/review-type sites. In some cases also news sources.
/claus
Froogle
That's it!>
Superscript - did you mean Scroogle?!
