|The State of the Internet (2013) - Summarised perspectives.|
My personal observations, and summary of information taken from a number of forums, and the personal observations of 20 or so webmasters with a combined traffic of 10s of millions of visits per day.
I've tried (my best) to summarise what I've seen in relation to this, and not editoralise too much.
Disclaimer to newer folk: Take this for what it is, a set of observations - somewhat muddied by our personal experience and expectations. We do not work for Google, we only have part of the picture.
A fairly clear pattern has emerged: The fight against spam was HUGE. At some level Google failed to react to the crossfire generated, and collateral damage. It hit widely, but no one more than UGC (User Generated Content). In part made possible by the narrowed gathering of, and seeming reluctance to consider, wider-perspective webmaster feedback.
Why all this UGC?
We are normally an order of magnitude bigger than everyone else. Things hit us first and more noticeably. Our area is 'gray', both quality and otherwise. We fight SPAM at the site level and tend to see updates before others do (due to our reach and size). In some regards, we may signal what's to come for SMEs (small and medium enterprises).
UGC site owners are hurting (pretty much across the board).
Even StackOverflow seems to have been hit in 2013 (according to alexa*)
"Too many updates, they're being careless." - 500 a year according to Matt.
WebmasterWorld (alexa* again) declining since 2011 (it's safe, don't worry, we all love it here!)
Reported: Perhaps a general decline in search traffic via Google (now at #2 spot on Alexa - Facebook #1)
Quality doesn't seem to be much of a factor. Many authorities hit hard.
No clear examples (so far) of older UGC that survived.
Examples of things like 'blank' sites, double H1, '2005 black hat SEO' making it into the top 10 (a lot of this seems to be done at the bottom of the page) Read the thread, it's quite entertaining*
Bing showing very different results.
Panda provided a boost for a lot of us UGC. I saw my traffic rise through 2011-2012. (21.5 nov 12 corrected that.)
(Unsure) Custom software, or updating your look may have resulted in another inadvertant penalty (*cross-fire, sticking your neck out)
Non-UGC: Manual adjustments for keyphrases patching and semi-correcting the problem here and there.
"Hard to say where traffic drops are occuring, it's just 'everywhere'"
Links are at the centre of this storm. UGC Webmasters forced to No-Follow everything meant organic user preference from the wider audience was lost. Hurts everyone.
The google product forums have unintentionally become an abysmal way to keep us from getting answers. (Our experience as we ventured in there recently to hunt down some answers)
Brands may not be favoured, but at this level of chaos - they're more likely to survive it.
According to some people: Black hat has become non-viable. If that's true, it may have been done by over-reaching on too many signal patterns.
I've left a lot out (paid links), as I haven't personally been following it. This is just my own meandering personal perspective.
* "Cross-fire collateral damage." In a time when bullets are flying around you, keep your head down. In web terms, that would mean blending in with the crowd. "Use traditional software, don't do anything custom-made. No-Follow every link." That's not a great situation, that hurts everyone.
* Alexa works pretty well (in my experience) for large sites - those of us in the top 10,000
* Some entertainment: [webmasterworld.com...]
Fully personal opinion
I'm a white hat SEO, for 15 years. First website made in 1994. IMHO: Good content has never been hit so hard, and to this extent. Some may argue, the current state of affairs will result in the loss of smaller deserving businesses, those without the deep pockets to survive turmoil to this degree.
IMHO Focus on content, even now. Nothing else has a longer-term chance of survival.
I don't believe any of this is intentional on Google's part, I believe them when they say they don't favour brands. PPC would be a better place for brands in general (they are used to that and can afford it, everyone wins). Leave organic to a fair mix of SME and brand - then the ad space used above the SERPs would be acceptable.
Question to the community
Do we need, as a community, to establish better dialogue with Google? If so, a proper way to do that. To keep the signal to noise ratio down. Real Googlers participating, hundreds (not 2). I believe some inside the web-spam team might agree, please speak up.
[edited by: goodroi at 1:18 pm (utc) on Jun 12, 2013]
[edit reason] per author's request, added question to the community [/edit]
Follow up: While Google apparently didn't respond to our questions or feedback, we're now seeing more and more of this in the press: [rawstory.com...]
If you go down the page a bit, you'll see (maybe the ? first ? time ? I've seen this in a circle outside of SEO) a mention of 'webmasters' being upset:
|Weinberg said another factor is that Google results are being gamed by search engine spammers and other companies trying to rank their results higher.” |
This article was FP reddit. Weinberg email was sent a few days ago (DDG) - it's on page 12 of this thread. Who knows, maybe he actually did read it.
I'm not saying we're the only ones talking about this obviously. But this whole thing has been mentioned in articles, videos and a few prominent blogs now. It seems to be gaining a little background traction. We done good.
Now can we get some dialogue?
> Now can we get some dialogue?
You did read the part where Google does 13.3 billion queries per month vs DDG measly 3 million, right? And how many of those Google queries do you think are from webmasters? I'd bet not enough for Google to care even a little bit about, if at all. But good luck to you in your quest to topple the mighty G anyhow. Who knows, maybe they are listening to your every word and running scared. Anything is possible, I guess.
@Play_Bach I think webmasters have more influence than you know. We are the content creators and Google does not survive without content to crawl. I don't think Google are running scared one bit. But also don't think there is as much Google Love out there either.
Google survive because in many countries there is no viable alternative. To most users Google provides satisfactory results but SME's and Content Websites have seen a rapid decline in their traffic. A specialist website is now replaced by brand or by a BH listing.
So we try to get some dialogue going, but that's ignored. I see our only alternative is to build our own search engine, and why not. If things are as bad as they seem for SME and Content Webmasters they should have plenty of time on their hands to create something new.
Hitchiker has tried very hard to get some dialogue going but it's fallen on deaf ears. But at least he tried. What would your suggestion be Play_Bach for a SME or Content driven website post penguin and panda?
|What would your suggestion be Play_Bach for a SME or Content driven website post penguin and panda? |
Short of government intervention, I don't see Google changing anything. Why should they? They're winning big time with things right as they are. There's not a search engine in sight that can compete with them, and they have the psyche of most of the people of the planet in their pockets. Until that changes, I don't expect they will either.
Government intervention did happen in Russia where Google's approach to buy Yandex was blocked by the Russian Government. But I don't see Europe or the US rushing to end the monopoly. Not whilst corporations are doing so well on the SERPS anyway. I don't see Government intervention ever happening to be honest.
One more *big* nail in Google's coffin ?... i certainly hope so!
'Privacy' search engines see jump after NSA row
Read more: [theage.com.au...]
It was a real pleasure forwarding this story to my entire newsletter list...
I really hope that one day Google will become too big for their boots. Their motto "don't be evil" makes me laugh. They may have had good intentions in the beginning, but no more.
I don't mean to share in other people's misery, but it is somewhat comforting to know that it's not just me who's taken a hammering. I initially thought my site had been hit by Panda, although had no idea why as it has good content that's unique & I've never engaged in black hat SEO, dodgy link building etc. But it looks like it's more a case of Google promoting advertising, their own products & big brands over and above the little guy. It is certainly NOT a quality issue, no matter what they say.
Today I "googled" another term related to my site & Yahoo answers was the first result. Again, I guess a bunch of 15 year olds have more knowlege than an expert on the topic.
Okay, duckduckgo, Yahoo & Bing all showed my article first. Google showed Yahoo answers (with several links to the topic below the first), another site that is completely copied from a book (I own the book), and then mine first.
I wonder if the authors of the book know?
|But it looks like it's more a case of Google promoting advertising, their own products & big brands over and above the little guy. It is certainly NOT a quality issue, no matter what they say. |
Yup. It's pretty obvious that's what's been going on, but especially so in recent months. First it was Froogle listings pushing down organics in 2006. Then came ads above and below organics, then Google places, Maps, YouTube, Google Shopping and brands. Now often ONLY ONE organic will show on a page, and that's if you're on a full screen desktop! On my iPad, I frequently don't see any organics on page one anymore.
More than anything else, I've long suspected the Google layout shift as being responsible for the big drop in traffic my site gets today versus Google 2009. Again, it's tough to make a buck if nobody comes in the store. Good luck everybody.
"All your base are belong to us"
My sentiments exactly. After spending the last week researching what exactly is going on at google I've realised too that government intervention is the only answer. Realistically this will take time, there are already some elements moving 'slowly' through the courts, and more in the making (anti-trust, etc) but it's going to take time.
The index is broken (I don't need to read all the reams of complaints on this site to know this, I can tell from my own sites, the new hastily compiled drivel ranks well, the long established, quality sites are out).
I love the analogy for the UK of the yellow pages, how could you operate a business if your business was listed one week, not listed for 4 months, then listed again under the wrong section? And that was the norm for 80% of the listings?
Under these circumstances, I can't employ staff, I can't forward plan, there is only one solution when google holds 80 - 90% of the UK search market...
I'm going BH.
This makes me very sad, but it's the Wild West out there and at the end of the day I'm working to support my family, they come first. I need to mine this current pot of gold while it lasts, and hope some day soon I can go back to my 'regular' business.
Good luck everybody.
|After spending the last week researching what exactly is going on at google I've realised too that government intervention is the only answer. Realistically this will take time, there are already some elements moving 'slowly' through the courts, and more in the making (anti-trust, etc) but it's going to take time. |
I would not expect too much from this. Governments have a difficult time telling any business what they can and can not display on their website. Coupled with the "partnership" that some governments have with Google, in regards to user data collection and dissemination, there is little incentive to go after Google because they choose to rank some sites instead of others.
Trust me, I'm not in Google's corner by any means. But the writing is on the wall. Free organic search traffic, for the most part, is coming to an end at Google. Although many webmasters are in denial about this, if you look at the trends you will see an expanding pay to play environment.
Will all due respect but how the hell does one build a BH site? I haven't a clue other than repeat keyword1 keyword2 keyword3 as many times in the titlebar and on page as possible ... or only mention it once with nothing else on the page ... or just get a load of spammy backlinks?
I'm not going to do it since I'm a branded, yes branded, group of companies, I'm fortunate in that I don't need Google to market my products, heavens knows what the scrapers do if they ever receive an enquiry for my widgets since so many of them are exclusive to me!
Realistically is The Net these days, according to Google, just a load of crappy sites actually doing nothing? Maybe some of the MFAers are making a few Dollars but I would have thought that the vast majority are doing next to nothing except frustrating the hell out of everyone here whose genuine sites and products cannot now be found because of all the garbage.
My main site is branded too, vehicle liveries, the works, I can't change the domain name without major upheaval - I will carry on maintaining the quality of that site for my customers, for traditional marketing activities and for other SE's. Google is irrelevant now to that site, I will not change it, or spend half the week chasing the forever growing list of bad backlinks to satisfy them.
To make up for the loss of traffic from G I will use the churn and burn tactics on new sites to pay the bills. If you can't beat 'em, join 'em.
|Yup. It's pretty obvious that's what's been going on, but especially so in recent months. First it was Froogle listings pushing down organics in 2006. Then came ads above and below organics, then Google places, Maps, YouTube, Google Shopping and brands. Now often ONLY ONE organic will show on a page, and that's if you're on a full screen desktop! On my iPad, I frequently don't see any organics on page one anymore. |
They've been boiling us slowly like a frog. Most visitors are little tree frogs and dead, we are bull frogs and still fighting.
|vehicle liveries, the works, I can't change the domain name without major upheaval |
Me too however over the past decade I have also bought up several keyword domain names gTLDs plus the same in a 4 letter acronym. Like you I'm leaving the branded name since that does extremely well in all the other SEs but I don't want to do a churn and burn therefore it looks like I'll have to use a completely different template construction to my own.
<off topic but relevant!>
I don't really want to go down the WP/Blogger/Joomla/Drupal route, apart from those has anyone found anything else really successful? I quite fancy the wiki style, any pros and cons with that?
|You did read the part where Google does 13.3 billion queries per month vs DDG measly 3 million, right? |
3 million A DAY is DDG's stats, not a month. Give them their dues. Up from 1 million a day in the previous month. Up from 2 million a day from last week. Who knows what their daily search numbers will be next week. They've grown so fast so recently due to the PRISM scandal. reddit's discussions are all about switching away from Google. Very interesting to read reddit fall out of love with Google - normally they're all over Google Glass, Fiber etc - but PRISM appears to be a thorn in Google's side.
Redditor here - Yeah great to see Reddit getting on this; which brings me to my next point:
I'm going to organise a little private group - one that is going to publish 'as-fairly-as-we-can-state-it' the truth of the implications, situation and best path forward.
We're going to upvote, comment and generally make as much of a PR impact as we can. Right now we're all wandering around commenting here and there.
If we don't do something, nobody will; it's that simple. This has gone far beyond ranking, this affects our planet and the voice of the people; not just a few webmasters on a specialised forum.
I now also know how to frame our little problem in a way that a much broader audience will appreciate.
These people will be WHs, with 10 mins to spare here and there. I'm off to go look for some now.
BTW: if you don't think this will work, you've haven't been on the internet long enough.
Keep us posted, Hitchhiker!
|Martin Ice Web|
Very interesting report of google positioning value. This shows the direct correlation between ads/google shopping inline organic seach and on sidebar. This is where traffic goes, this is why google puts ads in front of organic.
But this althought shows that google is wrong in avarage 67,5% for #1.
That's extremely useful, thanks. At this point research statistics are a breath of fresh air, keep 'em coming! :)
Somebody earlier pointed this out too..
Basically: By increasing PPC exposure above the organic SERPs, they've significantly reduced the amount of traffic going to organic results. It didn't happen 'overnight', it would seem to be a 'dial' that has been turned over a year or two. It explains a lot.
It would also explain why the 'traffic' is dropping from all sides (*everywhere) as mentioned in my OP.
Submerged beneath the Panda/Penguin 'effects' (both positive and negative), it would have been v. difficult for us to detect.
> reddit's discussions are all about switching away from Google.
@ColourOfSpring - I just went over there to see for myself. Where are all these discussions you speak of?
I really feel like they just turned on us. They used to be a pretty good company.
I have stopped working on all my websites. I have spent six months working my backside off adding new content, changing anything I thought may upset google, adding authorship etc., and it did absolutely nothing. So if I'm going to go bankrupt, I may as well do so reading a book &/or looking for a job, not slaving away on the sites. Where's the incentive to add new content?
I'm thinking of doing away with the Google authorship tool too. I felt like I had sold my soul to Google when I did that, but was prepared to do so if it meant more traffic. It's made absolutely no difference at all. I am sick of giving everything to Google on a platter.
I also feel they've mis-represented themselves. They have said they're trying to clean up the internet, while hitting good sites. You just have to look in this forum to see how many of us are working our butts off to make Google happy. When the reality looks like it's not actually about us and our sites doing anything wrong, it's about Google pushing paid advertisers and their own products. We're all chasing our tails for nothing.
> I also feel they've mis-represented themselves.
Search engine behavior conditioning 101
Step 1 - Top organic result is best choice
Step 2 - Civilians learn to click top result
Step 3 - Replace top results with ads that look like organics
2013 - Mission accomplished. That, my friends, is the reality. There's only so many pixels on a page and most of them aren't organic listings anymore. No surprise to me that my traffic has all but dried up - Google, Yahoo and Bing have it.
|You say yer life's a bum deal |
'N yer up against the wall...
Well, people, you ain't even got no
Deal at all
'Cause what they do
They just takes care
of NUMBER ONE
An' NUMBER ONE ain't YOU
You ain't even NUMBER TWO
-- Frank Zappa
Zappa had the mile high view even back then. One Adam 12, see the sucker...
|Martin Ice Web|
@Hitchhiker, and Google Shopping Counts for about 8! places. So there is nothing left for organics.
And you can bet that Google has some more detailed Information about ranking organics and earnings....
|It would also explain why the 'traffic' is dropping from all sides (*everywhere) as mentioned in my OP. |
But then they suggest it's our fault and have us running around like headless chickens trying to fix what's not broken.
Final warning from your overworked volunteer mods
This thread keeps disregarding the forum rules. We understand you are not happy with Google and have many valid reasons for feeling this way. Disregarding our forum rules and making extra work for volunteer mods who are struggling just like you to run our own online business is not the answer. If this disregard for forum rules continues this thread will have to be locked. Please behave :)
|@ColourOfSpring - I just went over there to see for myself. Where are all these discussions you speak of? |
play_bach, there are lots of threads on reddit regarding DuckDuckGo and Google in the /r/technology section - I did link to this but my post got removed.
@goodroi - Thanks for tolerating us this far, it seems most of this is fairly unprecedented. I spent about 3 pages of this thread trying to stop 'Google bashing'; the 'Google is evil' stuff has mostly died down.
This is: 'Google doesn't have our back - here's why - what do we do now?'
In essence: It may turn out that this is the beginning of one of the most devastating things to happen to SMEs in the 'story of online business'. We are trying to slowly the bits of it together - passion obviously runs high.
If this thread devolves into nonsense, we'll understand if it is locked. Sincere thanks for giving it exposure in the first place, and your patience thus far.
Problem is, the state of the internet DOES include the fact that Google is changing in ways that are unprecedented and in many cases inexplicable to those effected.
While it's true there is no point in JUST complaining/venting, it's helpful to identify these changes and discuss what they mean to us. And to get a consensus.
I think Google wants to disable SEO, and SEO/BH spammers are fighting back hard, and many of us are casualties in somebody else's war. That is to say: I don't think Google is out to destroy anyone's online business (why bother? Small businesses fail at a high rate at the best of times). I think they ARE out to make it so SEOs/BHs can't backward engineer the algo, and that is making it really hard for honest white hat webmasters to figure out how to rank.
But where it LOOKS like Google is either out to get us is when webmasters get outranked by their own scraped content, and stuff that egregious. But is that happening because Google is lousy, or because the SEOs and spammers are pretty bright people too, and they are fighting back hard?
As for the prominence of Adwords over the organic serps... I am totally sympathetic to those of you harmed by this, but... duh? It's their bread and butter, what did you expect? You can either build your sites around phrases that attract fewer ads, or start buying ads. Should the governments intervene? Maybe. Should public opinion hurt Google in the wake of PRISM? Maybe. But the reality is, Google is doing well WITH crappy SERPs, and that's the environment we have to try to survive/thrive in.
@Martin Ice Web
|This shows the direct correlation between ads/google shopping inline organic seach and on sidebar. This is where traffic goes, this is why google puts ads in front of organic. |
I'm not seeing this explained in the study. Am I just missing something? I'd love to know the overall percentage of visits to ads compared to organic.