The State of the Internet (2013) - Summarised perspectives.

[hitchhiker]

My personal observations, and summary of information taken from a number of forums, and the personal observations of 20 or so webmasters with a combined traffic of 10s of millions of visits per day.

I've tried (my best) to summarise what I've seen in relation to this, and not editoralise too much.

Disclaimer to newer folk: Take this for what it is, a set of observations - somewhat muddied by our personal experience and expectations. We do not work for Google, we only have part of the picture.

Summary

A fairly clear pattern has emerged: The fight against spam was HUGE. At some level Google failed to react to the crossfire generated, and collateral damage. It hit widely, but no one more than UGC (User Generated Content). In part made possible by the narrowed gathering of, and seeming reluctance to consider, wider-perspective webmaster feedback.

Why all this UGC?

We are normally an order of magnitude bigger than everyone else. Things hit us first and more noticeably. Our area is 'gray', both quality and otherwise. We fight SPAM at the site level and tend to see updates before others do (due to our reach and size). In some regards, we may signal what's to come for SMEs (small and medium enterprises).

UGC site owners are hurting (pretty much across the board).

Even StackOverflow seems to have been hit in 2013 (according to alexa*)

"Too many updates, they're being careless." - 500 a year according to Matt.

WebmasterWorld (alexa* again) declining since 2011 (it's safe, don't worry, we all love it here!)

Reported: Perhaps a general decline in search traffic via Google (now at #2 spot on Alexa - Facebook #1)

Quality doesn't seem to be much of a factor. Many authorities hit hard.

No clear examples (so far) of older UGC that survived.

Examples of things like 'blank' sites, double H1, '2005 black hat SEO' making it into the top 10 (a lot of this seems to be done at the bottom of the page) Read the thread, it's quite entertaining*

Bing showing very different results.

Panda provided a boost for a lot of us UGC. I saw my traffic rise through 2011-2012. (21.5 nov 12 corrected that.)

(Unsure) Custom software, or updating your look may have resulted in another inadvertant penalty (*cross-fire, sticking your neck out)

Non-UGC: Manual adjustments for keyphrases patching and semi-correcting the problem here and there.

"Hard to say where traffic drops are occuring, it's just 'everywhere'"

Links are at the centre of this storm. UGC Webmasters forced to No-Follow everything meant organic user preference from the wider audience was lost. Hurts everyone.

The google product forums have unintentionally become an abysmal way to keep us from getting answers. (Our experience as we ventured in there recently to hunt down some answers)

Brands may not be favoured, but at this level of chaos - they're more likely to survive it.

According to some people: Black hat has become non-viable. If that's true, it may have been done by over-reaching on too many signal patterns.

I've left a lot out (paid links), as I haven't personally been following it. This is just my own meandering personal perspective.

* "Cross-fire collateral damage." In a time when bullets are flying around you, keep your head down. In web terms, that would mean blending in with the crowd. "Use traditional software, don't do anything custom-made. No-Follow every link." That's not a great situation, that hurts everyone.

* Alexa works pretty well (in my experience) for large sites - those of us in the top 10,000

* Some entertainment: [webmasterworld.com...]

Fully personal opinion

I'm a white hat SEO, for 15 years. First website made in 1994. IMHO: Good content has never been hit so hard, and to this extent. Some may argue, the current state of affairs will result in the loss of smaller deserving businesses, those without the deep pockets to survive turmoil to this degree.

IMHO Focus on content, even now. Nothing else has a longer-term chance of survival.

I don't believe any of this is intentional on Google's part, I believe them when they say they don't favour brands. PPC would be a better place for brands in general (they are used to that and can afford it, everyone wins). Leave organic to a fair mix of SME and brand - then the ad space used above the SERPs would be acceptable.


Question to the community

---

Do we need, as a community, to establish better dialogue with Google? If so, a proper way to do that. To keep the signal to noise ratio down. Real Googlers participating, hundreds (not 2). I believe some inside the web-spam team might agree, please speak up.

---

[edited by: goodroi at 1:18 pm (utc) on Jun 12, 2013]
[edit reason] per author's request, added question to the community [/edit]

[Dymero]

@Martin Ice Web

This shows the direct correlation between ads/google shopping inline organic seach and on sidebar. This is where traffic goes, this is why google puts ads in front of organic.

I'm not seeing this explained in the study. Am I just missing something? I'd love to know the overall percentage of visits to ads compared to organic.

[hitchhiker]

The article shows the relative statistics for traffic sourced by various positions in a SERP. Therefore demonstrating the increase in traffic going to ads placed in the top positions.

[aakk9999]

The article shows the relative statistics for traffic sourced by various positions in a SERP. Therefore demonstrating the increase in traffic going to ads placed in the top positions.

I do not think the above conclusion (see bolded above) is correct. There is no information on ads anywhere - all that the report shows is a distribution of organic clicks.

This is not the distribution of ALL clicks (inclusive of ads positions and organic positions) and there is no information on the percentage of clicks that end up on ads.

If all percentages from the report are totalled up, the total is 95.2, the rest 4.8 percent is probably for SERPs positions 16 onwards.) Hence it is not known how the distribution would pan out if there were no ads. In fact, it is not even known how many of these searches (used for data collection) displayed ads and how many not.

This is not to say that ads do not influence the CTR to organic SERPs positions - however this cannot be seen/proven from chitika's chart.

To assert the "loss" in percentage of CTR of relative SERPs position, it would (for example) need to be known and compared the click distribution on SERPs that do not show ads versus the click distribution on SERPs where ads are shown.

[heisje]

24 June 2013 : EU asked to reject Google's offer in antitrust investigation

A nice bit of Official Google Bashing : Enjoy! [webmasterworld.com ]

.

[taberstruths]

Although it is not stated in the article, comparing the serp ctr percentages that are now happening vs. studies from 3-5 years ago, there is between 6% to 26% decrease in ctr percentage for the #1 position depending on the study you use. The lower positions are less affected but there has still been a decrease.

[Dymero]

But there was an increase in CTR for position 2 and 4. However, I suppose a shift to higher percentages on lower positions could be detrimental, as fewer people are likely to look at those even if they went up.

[aakk9999]

From what I can see, there is very small difference comparing with chikita study done in 2010 (3 years ago).

There is however a much bigger difference comparing with AOL data leak in 2006, so 7 years ago, but this was a different search engine with a different dataset (i.e. not google).

I am not aware of another SERPs percentage study - if there is, I would be interested to know.

[hitchhiker]

I do not think the above conclusion (see bolded above) is correct. There is no information on ads anywhere - all that the report shows is a distribution of organic clicks.

..this cannot be seen/proven from chitika's chart.

The article demonstrates a correlation between 'position' and 'effectiveness'. That correlation is, as expected, the lower down you are, the less you receive.

Q.E.D: Adding adverts above the organic SERPS is detrimental to organics. It does not give us a specific percentage, only a hint. (a percentage of people naturally ignore ads, so that would need to be factored in).

It would also be fair to assume that the more the saturation of advertising, the less traffic will go to organics.

It has been observed that the ratio of ads per page has been increasing steadily over the last year or so.

It would also be fair to assume that a fall in general traffic for many of us, and the now prevalent 3 block ads above many of our SERPS are related.

We have no conclusive facts, we are left to speculate based on observation and personal experience.

[rish3]

Also, if the paid results are more relevant to the query than the organic ones...

[GodLikeLotus]

Also, if the paid results are more relevant to the query than the organic ones...

+1

[seoskunk]

Are Google squeezing the life out of search whilst they still can? And taking every dollar they can. Recent statistics show the younger generation relying less on search and more social networks.

"Internet users ages 18 to 23, and 43 percent of users ages 24 to 32, used social networks as their go-to internet-discovery resource, according to a new report from Cambridge, Mass.-based Forrester Research"

"Overall, social networks like Facebook and Twitter are the preferred means of discovery for nearly a third of all Americans, up from 18 percent in 2010."

Source [m.entrepreneur.com...]

[aakk9999]

The article demonstrates a correlation between 'position' and 'effectiveness'. That correlation is, as expected, the lower down you are, the less you receive.

Oh, I agree with the correlation. But what we still do not know is whether the ad on the #1 would get the same percentage of clicks as if it would organic position #1 if there were no ads for the same query.

After all, ads have shorter titles and shorter descriptions, therefore less opportunity to convey the message the way you may wish.

Also, if the paid results are more relevant to the query than the organic ones...

Which is a factor too - and compounded with how bad the SERPs below is (or is not).

So whilst organic is definitely losing clicks because of ads, it is difficult to quantify how much and this we cannot see from chikita report. This is the data we are missing, as Dymero said:

I'd love to know the overall percentage of visits to ads compared to organic.

[aakk9999]

After the digression, to go back to the community question

Do we need, as a community, to establish better dialogue with Google?

What should we tell Google? I do not think the dialogue is so much a problem as what do we want from Google - to communicate what exactly?

[hitchhiker]

I do not think the dialogue is so much a problem...

I think you need to re-read what went on in this thread, we did give it an honest try for quite a while. We tweeted, emailed, G+'d etc. No replies came our way, and the next question on their youtube blog was about 'Albania'.

Saying that, fresh ideas are welcome. If you have a reason to believe you can get a question answered, you'll have support here.

What should we tell Google?

[webmasterworld.com...] - We had some ideas, examples here - consider adding something to it, we've not had many suggestions.

[johnhh]

Do we need, as a community, to establish better dialogue with Google?

You need two parties in any conversation, so let's turn that around...

Do we need, as Google Inc, to establish a better dialogue with the webmaster community on WebmasterWorld or elsewhere ?

Guess that gives you the degree of possibility to the first question...

[hitchhiker]

Guess that gives you the degree of possibility to the first question...

It was definitely a 'hopeful' proposition.

Considering the volume of feedback here, the 'fairly consistent' reports from hurt whitehats, SMEs, hobbyists and loyal netizens and the terrible hacks that are possible at the moment. This would be a great time to 'help us out' with addressing or, at least reviewing, our issues.

It would have been pretty hard for the Webspam team to miss this conversation. It's been linked to, mentioned and referenced in videos and a number of blogs.

Not sure what else we can do? Any suggestions?

[edited by: hitchhiker at 9:29 pm (utc) on Jun 25, 2013]

[johnhh]

Start a petition? list all the companies/individuals who have been affected, quote job losses ?

What you need is some little story that will get syndicated and turned into a big story mainstream.

Use a PR company ?

Actualy we had a jewelers here in the UK some years ago.The boss said they just sold cr*p, the whole chain went bust. Small story of his speech , massive effect.

[seoskunk]

Any suggestions?

A guest post from Snowden.......

[johnhh]

@seoskunk

don't think so - probably get tracked !

[ColourOfSpring]

aakk9999, here's some information for you to consider:-

[wordstream.com...]

Clicks on paid search listings beat out organic clicks by nearly a 2:1 margin for keywords with high commercial intent in the US. In other words, 64.6% of people click on Google Ads when they are looking to buy an item online!

Now, to be clear here, organic searches still get more clicks overall than paid search – but not all keyword searches are created equal. Keyword searches with high commercial intent – meaning, keywords where a searcher is looking to buy a product or service (for example: “buy stainless steel dishwasher”) – are worth far more to businesses than your basic informational keyword searches (for example: “who is Thomas Edison”). Our research found that for these valuable, high commercial intent keyword searches, paid search advertising listings gave the “free” organic search listings a resounding beat-down.

Survey Methodology

Our survey was limited to advertisers in the US, for Google Search only. In our survey, we define high commercial intent keywords specifically as keyword searches on Google that have significant advertiser competition and trigger a Google Shopping or Google Product Listing ad.

We used recent average click-through rate data collected through our AdWords Performance Grader across over one thousand AdWords accounts in the last 60 days. We also looked at the Google Analytics/Webmaster Tools and AdWords account data of WordStream’s managed accounts to analyze organic click-through rate data and trends.

Because people often click on multiple ads and/or organic search listings from a single search result page (which makes the click-through rates of all of the paid and organic listings add up to more than 100%), we normalized the CTR data to reflect the % share of traffic generated for each paid and organic search listing present on a typical search engine results page.

[aakk9999]

@hitchhiker
I have in fact read this thread and for a while it kind of turned into a whinge about Google which kind of sidetracked us from the great OP that you started this thread with.

I have also read the other thread you have mentioned in your post above [webmasterworld.com ] which is, as you say, very short - indicating people are struggling to come with more constructive questions on what to ask.

The problem that I am seeing is that many things we would like to ask Google to disclose can be (and will be) used by black hats as well. We cannot say "we are the white hats, tell us" because unfortunately it does not work this way.

A number of times in the past Google reviewer guidelines have been leaked out and these should tell us how Google sees/thinks what a good site is. The difference between black hat and white hat is that white hat would strive to build such site, whilst black hat would like to emulate the signals such site sends.

I am struggling to see on how can Google give answers to white hats and at the same time ensure this answer is not used by black hats to their advantage and SERPs manipulation.

[kellyman]

Google is a mind set, its a habit, if you have not already then you need to break that habit,

I bet people complaining of Google are still using Google, I do still out of habit, the only ever way Google will change if they see any kind of threat, there is no real threat at present, and for every web-master pissed with the search results there's another web-master got great results who took their place.

I honestly believe there has been so much tinkering with the algo it cannot be fixed without a massive overhaul of the system, search is changing, but im not quite sure its social just yet

[Saffron]

I bet people complaining of Google are still using Google

Nope. I don't use Google anymore.

for every web-master pissed with the search results there's another web-master got great results who took their place.

My site did great from the original Panda. Traffic increased dramatically. It's down 50% this year. Nothing changed on my site. So what made it so popular with Google two years ago and so unpopular now?

[kellyman]

My site gained through all the panda updates until the very last one, lost 90%

Was thinking the same thing how can it turn so drastic, have no idea where to turn or what to do, need to earn a living and now paying adwords (they got me) even hired a telesales person to keep going,

Have to sit and wait and see what happens as we sure cannot make it happen anymore.

[mrguy]

I bet people complaining of Google are still using Google, I do still out of habit, the only ever way Google will change if they see any kind of threat, there is no real threat at present, and for every web-master pissed with the search results there's another web-master got great results who took their place.

I stopped using them a long time ago and have never missed them. I also block my PC from being cookied by them or any of their ad networks for the rare occasion I end up on them doing research for a client.

I went so far as to dump my droid as well and got an iPhone. I don't care if Apple tracks me, at least they are not Google. LOL

[taberstruths]

I haven't used google for search for over a year with the exception of very long tail phrases. Sometimes Bing can't handle 4+ word searches.

[hitchhiker]

My site gained through all the panda updates until the very last one, lost 90%

Yep, I was up and up throughout Panda till Nov 17th 2012. Now I'm down 80k people a day. My competition are basically gone.

there is no real threat at present

Well, actually, there is.. there's a growing need for diversity in search for a number of reasons. People are becoming more aware of that in light of other developments.

The problem that I am seeing is that many things we would like to ask Google to disclose can be (and will be) used by black hats as well. We cannot say "we are the white hats, tell us" because unfortunately it does not work this way.

@aakk9999 thanks for helping keep this thread on track - much appreciated. That is a fair point, except for one thing: Blackhats, by nature can thrive with hidden penalties infinitely where we can't.

If we're all in the dark + penalties exist: (without penalties nothing below is true)

Those willing to fire at anyone, with throwaway sites, and no commitments will stumble upon penalties, restart, restart, restart and eventually stumble upon success.

Those with 'reputation' to consider, will eventually stumble into penalties, and not be able to 'throw away' their sites. Game over basically.

Blackhats try everything anyway, they don't listen to the guidelines as we do - they're smart as hell, they guess what Google are doing as easily as anyone else.

TL;DR; the less information the more likely blackhats will take the advantage as they can afford to recycle after hidden penalties, we can't.

[HuskyPup]

Ok, a bit of an Englishism here ... Google's cream-crackered, they are absolutely fubard with what they have created.

Those who are still relying on Google to deliver sensible and clean results are wasting their time, it ain't going to happen any time some and possibly never.

Dialogue will never happen with us, they have enough internal strife over what has been created and what they "think" they can do with the "knowledge engine" by leeching off all sites as much as possible for as long as they can.

In many companies they'll not give a flying f++k whether Google leeches or not, overall personally I don't since Google knows nothing, zilch, denada, niente about my widget industry and its patheticness is displayed in the SERPs, no one in my industry trusts Google to deliver any sensible result.

However, the issue for many site owners is whether or not they can continue to make a reasonable living solely out of websites with no exclusive realworld products or information?

The scrapers and the spammers are absolutely NO different to Google, they're trying to justify themselves as realworld businesses when, in fact, they are nothing of the kind. Is it any surprise that Blogger sites do so well? Hey kidddes, create a crap site, stick some ads on it and we'll enjoy the benefits together.

This entire G thing is the biggest pyramid scam in years, sure there are many genuine sites out there but we're WAY out-numbered by G's very own spamming & scamming system.

</not a rant, maybe a revelation for many?>

[mrguy]

However, the issue for many site owners is whether or not they can continue to make a reasonable living solely out of websites with no exclusive realworld products or information?

That's a key statement!

Not a single client I do work for that has their own products or service has suffered in any of the Google updates even up to today. Sure, they shuffle a few here and there, but they remain solid. They are established and have branding within their industry.

Other's I work with who are more or less affiliates, are having a hard time.

Even if you're a re-seller, in Google's eyes that's an affiliate and they hate affiliates with a capital H.

[hitchhiker]

Internet != Your client's types of sites only.

It includes everyone from hobbyists to forums, special interests, education, commentary and a hundred thousand others. Some of us are not selling a product, don't need the services of an SEO.

Yes, branding can work for a lot of businesses - awesome, completely agreed on, totally great!

Everyone else should also just brand up? - and once they've inappropriately tried to stuff themselves into that box; what quality will that have created?

Also, a lot of people seem to prefer an internet that doesn't solely consist of 'brands'. Some of us think that would be a bad situation for everyone. A lot of quality can be found in independent retailers who've focused on their product more than their branding - they shouldn't be dismissed so easily.