Forum Moderators: Robert Charlton & goodroi
Perhaps, based on Matts recent post [mattcutts.com] where he says:
....expect those (roughly monthly) updates to become more of a daily thing. That data refresh became more frequent (roughly daily instead of every 3-4 weeks or so)....
it is time to coin a new term, EVERFRESH. Google, it's full of Everfreshness. :)
---------------------------------
Seriously though, let's take Matt at his word (if you find that difficult then for a moment at least) when he says:
I know for a fact that there haven’t been any major algorithm updates to our scoring in the last few days, and I believe the only data refreshes have been normal (index updates).
Instead of saying Google is broke why don't we ask ourselves what could be the causes that produce SERP fluctuations or the reports thereof?
For example:
Now that we are in the EverFresh era with daily updates I expect that SERP changes have become more obvious for the exact opposite reason that movement in film becomes more pronounced when you slow down the rate at which frames advance. We have been watching the length of time between SERP changes shorten with each Google upgrade. We have also witnessed, over the years, communication between the data centers improve and accelerate. Now, these have combined and moved so close to real time (when compared to the Google Dance days) that data center watching has become virtually meaningless over the last six months. (The Google Dance has become a GoogleRave.)
I’m not saying that Google never sticks a hot iron poker into their works (and I definitely think they have made some silly choices like, ahem, their no-follow policy). It’s just that there are far more, a near infinite percentage more, changes happening to the content and population of the documents on the Internet. I expect to see lots of change to the SERP results.
So, assuming that Matt is as honest as he is nice, what do you see as non-Google causes that can be responsible for noticeable SERP fluctuations?
[edited by: tedster at 3:23 pm (utc) on Dec. 25, 2006]
Googlebot is working over time but SERPs for my sector seems to be from a differant planet at this point. search terms w/or w/o quotes give far less relevant results than before Dec 20. Also, the "short" URL issue seems to be a factor. Listings start w/a short URL and then get longer as I work thru the results. (I never paid any attention to it before, but it seems to be a factor now).
As far as I can tell google has changed something (whether they know it or not), and it's not anything on our end of the WWW.
What gives me a little hope is that the "Query stats" in Googles "Webmaster Tools" still show me good rankings as before.
I know that this tool is normaly delayed but not more than 7 days!
So what's going on there?
Hi Folks
First off, the followings is by no mean established facts. Its rather an honest attempt to understand the recent shift in Google’s Updates Policy.
December 2006 hasn’t been the most happy month of the year for several fellow WebmasterWorld members. For obvious reasons we don’t have an exact number of affected sites. However, there is no doubt by now that those Google “savage” Data Refreshes have affected ranking, indexing, backlinks, traffic and ultimately the business of several members (as expressed through many related threads).
Though Google assured us (through our kind fellow member Matt Cutts) that there haven’t been a “major” algo update or “major changes”, there is no doubt whatsoever that the impact of those Data Refreshes resemble the impact of the classical updates as Allegra and Bourbon etc..
Because some of us don't wish to consider those Data Refreshes as major algo updates, many find it very hard to understand why their sites have been affected (sometimes very badly).
Reading the feedback of affected fellow members on the different threads, It seems that Google has hit again very hard in this December on Duplicates or what it regards as duplicates or Unworthy Contents.
It seems that the contemporary Algo Updates function in stages through successive Data Refreshes.:
- Identify the Unworthy Contents
- Deprive the Unworthy Content of its rankings either by “converting” it into supplemental or just de-index it.
In that connection, I respectfully disagree with Matt Cutts that “supplemental results by themselves don’t indicate badness/penalties/problems”. IMO, Google treats supplementals as Unworthy Contents which is kept isolated in its own Unworthy Index.
- Depending on the volume of Unworthy Content, the entire site might loss part/big part of its Worthiness and decline in rankings in general.
IMO, one of the symptoms of sites suffering of decline in Worthiness might be illustrated through site: operator. While site: operator displays what we call “normal” listing (most important pages at top of the list) for a “Worthy” site, the same site: operator might display supplementals at top of the listing or displays no “logical” order in listing for affected sites (less Worthy or Unworthy).
I ‘m looking forward to hear more details (case history if possible) especially from kind fellow members who have been affected by what it seems a “Duplicates-Penalty” during the so called “Data Refreshes” of December 2006.
Thanks!
[edited by: SEOPTI at 11:44 am (utc) on Dec. 29, 2006]
I see that "Unworthy Content" gets crawled heavily.
Unfortunately, getting crawled heavily does not necessarily mean getting out of supplementals, nor does it guarantee a listing in the index that can be found by searchers for any of the keywords or phrases. So, for all logical purposes, the page isn't indexed.
I don't know of many searchers who will dig to #700+ in an attempt to find what they're looking for. Perhaps a few will, but not enough for a successful website.
They just seem to have so many filters and penalties that they are tripping over themselves and catching everything in sight, except of course for the big guys like wiki, who have a 'get out of jail' card, however useless their content on a topic.
Once silver lining though is that this forced me to use live.com for the first time. And hey, it isn't at all bad. It showed me that mine isn't the only worthy site missing from Google.
Almost all of my fresh content is still indexed, and is non-supplementary. Those pages that are supplementary have been supplementary for quite some time, and this never effected my rankings. My ranking started tanking in early December, and have been obliterated since December 20th. I haven't been banned, just decimated in rankings, like never before.
The site:www.domain.com operator yields the same # of pages that it always did, but my Supplementals are now listed first.
In my specific case, there are a few pages that rank high for certain keyword terms, but most of my major terms are gone for my main pages. My pages contain unique content, unless they have been duped by others. Specifically, I have used COPYSCAPE against a number of my pages, that return NO results (hence, not duplicated). These pages won't even come up in the SERPs when I enter the page title or description verbatim in my searches (they do, however, when they are placed in quotes - and even then, show up on Page 2 mostly). Yet for several other pages, placing my title in search verbatim, puts me at the top of the heap. This inconsistency allows me to cling to the hope of a "bad data push", as it is not fully consistent with the negativity mentioned above.
One additional observation that I noted today, was an error that was listed in my Sitemap diagnostic page. To reiterate from my prior posts, I have 9 websites that all fall under my single domain (has been in existence for 10 years). Each of the 9 websites are contained in a subdirectory. I have been ranking well for each for many years. The format of my subdirectories are:
www.domain.com/subdir1/home.htm
www.domain.com/subdir2/home.htm
.
.
etc.
Of the 9 websites, I consider one to be MAJOR, since it receives the most traffic, due to its searchable content.
SITEMAP OBSERVATION: On December 15th, I have a Sitemap Diagnostic error that states that:
www.domain.com/subdir1 yielded an http error (403 - Forbidden). Note that this isn't a "network unavailable", but a 403 on a specific directory (not a page). I do have (and always have had) directory listings disabled on my webserver, but I found it odd that my entire directory (for this heavy-hitting website) came up 403 by googlebot. For fun, I enabled directory listings, just to see if this had anything to with my ranking tankings.
Additional sitemap observations that seem a tad weird:
- I have 21 Not Found sites listed in diagnostic of late, all pointing
to subdirectory/page combinations that don't exist. To verify this,
I spidered my site using a tool that I have, and none of these
21 alleged unfound pages, turned up as being referenced on my
site
- I have 6 unreachable pages noted from Dec 14 - Dec 26th. My
network (I host my own site) has not been down at all during
that time, but I guess it is possible that my provider wasn't
sending things through during these times. Of note, 2 of these
6 warnings, mention that my robots.txt file is missing. I don't
have a robots.txt file for any of my sites, but this is the 1st
time, I've ever seen this error logged.
QUESTION # 1: For those who have been uneffected by this Dec. 2006 phenomenon, do your Supplementals show up first?
QUESTION # 2: For those who have been effected by this December tornado, have you seen anything similar in your Sitemap diagnostics?
[edited by: doughayman at 1:52 pm (utc) on Dec. 29, 2006]
It seems very strange that some search terms are unaffected while most have proportionally sunk, leaving the overall popularity of pages in the same order and almost the same keyword popularity order.... go figure?! This seems like a datapush error rather than any deliberate change. In the past you could see a pattern... perhaps longtail searches lost or specific keywords lost. There is only one pattern for us and that is one major page has lost all its rankings, the rest have all just proportionally dropped.
Yes, I agree. The inconsistency alludes to error of some sort. If things were consistent across the board, I could accept algo change, but they are not.
MHES (and all), please re-read my post above (which I just edited), to read about my querky Sitemap error findings.
Regards,
Doug
The site:www.domain.com operator yields the same # of pages that it always did, but my Supplementals are now listed first.
Thanks for feedback. That looks like the symptom I talked about in my previous post:
IMO, one of the symptoms of sites suffering of decline in Worthiness might be illustrated through site: operator. While site: operator displays what we call “normal” listing (most important pages at top of the list) for a “Worthy” site, the same site: operator might display supplementals at top of the listing or displays no “logical” order in listing for affected sites (less Worthy or Unworthy).
[edited by: reseller at 2:00 pm (utc) on Dec. 29, 2006]
Thanks for your feedback. Specifically, are you seeing a 403 (forbidden) logged in your Sitemap Diagnostics, being applied to a root directory as opposed to a page?
Well there could be a good reason. Google has suffered from bad publicity in the past with big updates. Often it does not matter if the update is good or bad, there have been enough complaints to warrant ill feeling which has been picked up by the press. By doing these changes in small doses, there is never enough impact to affect their share price, especially if it goes wrong. Members here report the same observations going back months, if I was google, I would do the same and roll out changes in small pockets below the press radar.
Anybody else seeing the same order for busiest pages before and after the drop? This could be a clue to where we should be looking.
[edited by: MHes at 2:40 pm (utc) on Dec. 29, 2006]
- For my major keyword terms, the top results seem to be mostly
root page domains, as opposed to specific pages within the
domains
- For one of my sites' subject domains, which has previously
infested with spam, most of the spam seems to be gone. The
specific "spammer team" responsible for this spam (it was
obvious that it was being generated from the same guy), his
former spam pages were black-hat deposits on forum sites that
had obvious holes, for allowing such garbage. The URL for
these pages typically were pages many subidrectories deep. For
example:
www.domain.com/subdir1/forum/post/gobbledeegook.htm
Is it possible, and I'm just throwing this one on the table, that this Google update may have been a major spam filter, that somehow went bad?
Are those that are being effected, missing results from the SERPs, that were pages buried several directory levels deep under their root domain? I fit this profile, and just a weak theory that I'm throwing out on the table.............
Observation = the further from the home page, the bigger the drop in traffic compared to last month.
When did they stop saying "your search contains the words 'in' and 'from'...." at the top of the serps? Is this new?
[edited by: MHes at 2:54 pm (utc) on Dec. 29, 2006]
I just answered one of my own questions above. For a subject domain that I normally ranked high for (and now don't), the sites that come up first on the list, also have their supplemental pages come up first (if they have supplmentals), when I do a:
site:www.domain.com
Hence, having your supplementals come up first does not appear to be a symptom of being effected by the December obliteration.
Further to my post above, and to yours, the terms that I am still ranking high for, are very "short" pages names like:
www.domain.com/subdir1/abc.htm
Keep in mind that this may be a ludicrous assertion. Also, note that all my pages (even my root website pages) start at Level 2.
Could there be some sort of punishment based on level? That is, pages that are further down in the tree are being deemed as less important?
This would be a ludicrous imposition by Google, unless it is an attempt to rid their indices of spam, and this filter went awry.
NOT SURE -> "When did they stop saying "your search contains the words 'in' and 'from'...." at the top of the serps? Is this new?"
Thanks reseller. Can I assume then (assuming that you have been uneffected), that your "site:www.domain.com" command lists your supplementals last, as opposed to first then?
Of course we can't issue a general judgement. However I would assume that a site with entirely "Worthy Contents", in most cases, is that which shows “normal” listing whith no supplementals for a site: search.
In the case of my kind WebmasterWorld fellow member Graywolf website [mattcutts.com], the site: search shows supplementals at the end of listings. Graywolf has posted on this thread [webmasterworld.com] a comment about that:
I'll say that the order the pages are listed for the [site:] command is still a bit screwy. True the supplemental results aren't first, but the pages listed in the top 10 aren't pages that get the most traffic or have the most inbound links. If you are super curious the 'top posts' section gives you a much better idea of what's linked to and getting the most traffic. I do plan to do a little testing on the "short url" factor.
[edited by: reseller at 4:04 pm (utc) on Dec. 29, 2006]
I agree. I don't think Google spends as much time worrying about the site: command as webmasters do. When I look at this command I do see one supplemental on page 1. But that particular page is nearly an "orphaned" page. In this case it's a spreadsheet that can be downloaded and it's used as a supplementary tool for an article. So I'm not surprised it's supplemental because it only has one incoming link that I know of and it serves a very specific purpose.
Now if my home page came up supplemental, then that would worry me. Other than that, I'm taking Matt Cutts at his word and not worrying too much about supplementals.
Now if my home page came up supplemental, then that would worry me.
But I assume that it will worry you much if other "important" pages of your site, which you wish them to rank, will be turned supplementals, right?
Not sure if I was clear on this or not, but my pages that are supplemental are either orphaned pages, or pages that are 6-8 years old and haven't been updated in ages. In addition to my "business" websites under my root domain, I also have some personal websites and some old sites that have gone supplemental. I have made no attempt (nor do I desire) to get these pages out of supplemental purgatory.
ALL of my key pages are non-supplemental, and that hasn't changed through this December debacle. Other than the supplemental pages listing first on my "site:" command, I don't see any relevance associated to this right now, especially since I've seen some very high-ranking sites (currently) list their supplementals first.
Regards,
Doug
Hmmmm... not convinced. None of my pages have suddenly become supplemental but rank below them on site:search. However, one of my pages could have been one of the sites you saw rank number 1 ....... and I have been effected.
Is it possible, and I'm just throwing this one on the table, that this Google update may have been a major spam filter, that somehow went bad?
I think it's a long tail problem that had its first phase in the October massacre:
[webmasterworld.com...]
The problem is that the long tail is thinly populated, and so when attempting to automate the removal of spam (which mostly resides in the long tail) it's easy to do a lot of collateral damage. We are the victims of that collateral damage.
However, there may be two, three or four different issues all running in this thread....sigh!
Not necessarily. Going back to my example. I've got this supplemental on page 1 of my site: command results - and that's a spreadsheet. If that was my home page I'd be worried because my home page is what my website is all about...
But if I used the "spreadsheet site: www.mydomain.tld" command I see that supplemental page and it's no longer listed as supplemental. My conclusion is that this particular page is not a good result for the simple site: command but it's a good result for the term "spreadsheet." In fact, I've got other pages that show as supplemental that rank fairly well for the term they're targeting.
< This discusion continues here: [webmasterworld.com...] >
