The State of the Internet (2013) - Summarised perspectives.

[hitchhiker]

My personal observations, and summary of information taken from a number of forums, and the personal observations of 20 or so webmasters with a combined traffic of 10s of millions of visits per day.

I've tried (my best) to summarise what I've seen in relation to this, and not editoralise too much.

Disclaimer to newer folk: Take this for what it is, a set of observations - somewhat muddied by our personal experience and expectations. We do not work for Google, we only have part of the picture.

Summary

A fairly clear pattern has emerged: The fight against spam was HUGE. At some level Google failed to react to the crossfire generated, and collateral damage. It hit widely, but no one more than UGC (User Generated Content). In part made possible by the narrowed gathering of, and seeming reluctance to consider, wider-perspective webmaster feedback.

Why all this UGC?

We are normally an order of magnitude bigger than everyone else. Things hit us first and more noticeably. Our area is 'gray', both quality and otherwise. We fight SPAM at the site level and tend to see updates before others do (due to our reach and size). In some regards, we may signal what's to come for SMEs (small and medium enterprises).

UGC site owners are hurting (pretty much across the board).

Even StackOverflow seems to have been hit in 2013 (according to alexa*)

"Too many updates, they're being careless." - 500 a year according to Matt.

WebmasterWorld (alexa* again) declining since 2011 (it's safe, don't worry, we all love it here!)

Reported: Perhaps a general decline in search traffic via Google (now at #2 spot on Alexa - Facebook #1)

Quality doesn't seem to be much of a factor. Many authorities hit hard.

No clear examples (so far) of older UGC that survived.

Examples of things like 'blank' sites, double H1, '2005 black hat SEO' making it into the top 10 (a lot of this seems to be done at the bottom of the page) Read the thread, it's quite entertaining*

Bing showing very different results.

Panda provided a boost for a lot of us UGC. I saw my traffic rise through 2011-2012. (21.5 nov 12 corrected that.)

(Unsure) Custom software, or updating your look may have resulted in another inadvertant penalty (*cross-fire, sticking your neck out)

Non-UGC: Manual adjustments for keyphrases patching and semi-correcting the problem here and there.

"Hard to say where traffic drops are occuring, it's just 'everywhere'"

Links are at the centre of this storm. UGC Webmasters forced to No-Follow everything meant organic user preference from the wider audience was lost. Hurts everyone.

The google product forums have unintentionally become an abysmal way to keep us from getting answers. (Our experience as we ventured in there recently to hunt down some answers)

Brands may not be favoured, but at this level of chaos - they're more likely to survive it.

According to some people: Black hat has become non-viable. If that's true, it may have been done by over-reaching on too many signal patterns.

I've left a lot out (paid links), as I haven't personally been following it. This is just my own meandering personal perspective.

* "Cross-fire collateral damage." In a time when bullets are flying around you, keep your head down. In web terms, that would mean blending in with the crowd. "Use traditional software, don't do anything custom-made. No-Follow every link." That's not a great situation, that hurts everyone.

* Alexa works pretty well (in my experience) for large sites - those of us in the top 10,000

* Some entertainment: [webmasterworld.com...]

Fully personal opinion

I'm a white hat SEO, for 15 years. First website made in 1994. IMHO: Good content has never been hit so hard, and to this extent. Some may argue, the current state of affairs will result in the loss of smaller deserving businesses, those without the deep pockets to survive turmoil to this degree.

IMHO Focus on content, even now. Nothing else has a longer-term chance of survival.

I don't believe any of this is intentional on Google's part, I believe them when they say they don't favour brands. PPC would be a better place for brands in general (they are used to that and can afford it, everyone wins). Leave organic to a fair mix of SME and brand - then the ad space used above the SERPs would be acceptable.


Question to the community

---

Do we need, as a community, to establish better dialogue with Google? If so, a proper way to do that. To keep the signal to noise ratio down. Real Googlers participating, hundreds (not 2). I believe some inside the web-spam team might agree, please speak up.

---

[edited by: goodroi at 1:18 pm (utc) on Jun 12, 2013]
[edit reason] per author's request, added question to the community [/edit]

[hitchhiker]

Hey rish,

I may be mistaken

It seems I was mistaken.

Either way, we're not going to get communication if we don't focus a call for it it somewhere. This thread is that attempt.

[rish3]

Going the extra level of detail here.

Draw your own conclusions on Panda/Penguin, and their effect on Revenue and Click growth.

Quarter Rev Clicks Events

Q1 2011 +27% +18% Panda 1.0(Feb)
Q2 2011 +32% +18% Panda 2.0(Apr)
Q3 2011 +22% +20% Panda 2.2(Jun), Panda 2.3 (Jul)
Q4 2011 +25% +34% Panda 2.5(Sep), Panda 3.1 (Nov)

Q1 2012 +24% +39% Panda 3.2(Jan), Panda 3.4(Mar), Above-Fold (Jan)
Q2 2012 +21% +42% Panda 3.5+3.6(Apr), Penguin 1.0(Apr), Penguin 1.1(May), Panda 3.7+3.8 June
Q3 2012 +31% +20% Panda 3.91(Aug)
Q4 2012 +36% +24% Penguin Refresh(Oct), Panda Refresh(Nov) Panda Update(Dec)
Q1 2013 +22% +20% Panda Update(Jan)


You see any hits to revenue or click growth? I don't :)

For comparison, here's 2010's numbers, prior to Panda/Penguin. You can see the YoY growth numbers are smaller.

Q1 2010 +21% +15%
Q2 2010 +24% +15%
Q3 2010 +23% +16%
Q4 2010 +26% +18%

[heisje]

Or, in short, the truth seems to be the exact opposite of what Matt said.

Surprise, surprise . . . .

.

[heisje]

if (stating-facts) = $GoogleBashing && $GoogleBashing != Helpful;
{
(stating-facts) != Helpful;
}


Unwittingly playing Google's game? By all means, if that makes people feel better.


And, how do they run away with it?
PRISM (and such) anybody?


This is "The State of the Internet (2013)".
(quite on topic, sad to say).



.

[Awarn]

@hitchhiker

I believe you understand the power that webmasters here could control, however you might be going about it in a way that may actually not be best for most of us. We don't want to be at the mercy of Google or Bing or anyone else. Think more along the lines of Grover Norquist (not that I think that stance is exactly correct). It does show that a united front can bring even the most powerful to their knees.

[turbocharged]

Either way, we're not going to get communication if we don't focus a call for it it somewhere. This thread is that attempt.

While a noble effort, the PRISM scandal and questionable past statements made by Google will likely cause them to limit public discussions in the short-term. I would suspect a public relations effort to repair Google's damaged credibility is on the horizon as what has transpired in and out of the SERPS has significantly diminished confidence in their company. Webmasters AND users now have reason to question Google, and that is something I'm sure they would rather play out in a controlled environment where rebuttal is not possible.

[diberry]

I've been puzzling over Matt's comment: why lie about something that's so easy to disprove, that so many of us already knew wasn't true because we follow G's revenue statements closely? He could've just said something like "Panda cost us money". I'd be very interested to hear his answer to why he said this.

[seoskunk]

Webmasters AND users now have reason to question Google, and that is something I'm sure they would rather play out in a controlled environment where rebuttal is not possible.

A very accurate observation. I do not think Google will be able to spin this story (PRISM) if open to discussion. They will want to make it hard to respond and feed out controlled propaganda. In short I see no benefit for Google to open up a line of communication with webmasters. There is no financial incentive and it could turn into another PR disaster on top of recent disclosures.

A noble effort hitchiker for sure but don't expect the re-emergence of Google Guy anytime soon.

[Mac_Guy]

If the general public, the ones that uses G to buy products or search information on the internet would know just half of what we webmasters or small to medium
website business owners knew, they would use another search engine, who wants to have a returned search that is manipulated or decides for you where to direct you inquiry ?

[heisje]

I wouldn't be surprised if the train of events were as follows: a few years ago Google et al. were "blackmailed" in PRISM and other such activities (yet undisclosed) on account of potential Anti Trust action. Once in, they got a free ride in the marketplace, siphoning money from your pocket into their pocket, in ways that otherwise could be challenged as of dubious legality (abuse of market dominance, and much more). The real economy did not grow 30% per quarter - their bank account did. Anti Trust investigations were virtually dropped.

Historic timing supports totally this train of thought - however we will never know.
Is worse yet to come? Time will show.

As about Cutts lying, arrogance has made them careless. No mystery there.
.

[hitchhiker]

As somebody who's been actively aware, learning, discussing and educating people about projects like PRISM for the last 15 years can I just say: THIS is not the place for that discussion. You can safely start another thread about it somewhere else.

I've spent a lot of time, a LOT of time, getting this information into one place. Please respect that enough to not flood this thread, and bury it under a mountain of noise.

Meanwhile: [news.ycombinator.com...] (Matt posts yesterday, my reply)

The original post is: [news.ycombinator.com...] - A googler since 2006 wrote it apparently. Barry Schwartz picked up on it.

I couldn't agree more, he's absolutely spot on. This whole thing seems to be picking up steam.

[ColourOfSpring]

If Google really wanted to listen to us, Matt Cutts videos wouldn't exist. They exist BECAUSE Google don't want to listen to us. His videos are a public relations exercise. They're a carefully controlled conduit between Google and webmasters. They frame the debate and carefully ignore a lot of real concerns we have. If Google TRULY wanted to work with webmasters, we wouldn't be listening to vague statements in videos and trying to decipher them. I honestly can't see Google changing this direction in the near future unless they start hurting financially. If you check their share price, that ain't gonna happen for a few years yet at least. The only way to deal with Google is to either build a business model outside of Google (which I continue to do) or you campaign AGAINST Google - promote other search engines and traffic sources outside of Google, highlight their shortcomings, basically choose to NOT work with Google since they most definitely do not work with or care about you (that is not even open to debate).

[Mac_Guy]

@ ColourOfSpring:

I agree 100% with your posting.

"promote other search engines and traffic sources outside of Google" 200% on that one!

Can someone name one publicly traded company that is not there to reward shareholders?

[dvduval]

When you look at "webmaster" related sites on Alexa. These are all down from Penguin 2 and the update before that:
Wordpress (com and org)
Vbulletin and Xenforo
Hotscripts
DP and Warrior forums

Then you look at companies like Amazon, Expedia and Google content sites and they are up.

I remember speaking to a member of the Adwords team about the organic serps mostly being below the fold, and the response was, "but don't you find the ads a lot more relevant now?"

I would say they are relevant when it comes to corporate profits. They are relevant when a small group of people want to convince a larger group that their way is the best way.

"Reverence" to the corporate culture rather than "relevance" of the best result... Reverence is the word of 2013, not relevance.

[hitchhiker]

This is why we can't have nice things.

"Thread about communication with Google; takes weeks to research"

"Now is Yet-Another-Google-Is-Evil thread, no new information, no new findings and nothing to help failing sites; will be forgotten by all in a few days."

Cheers. Nice work.

[ColourOfSpring]

hitchhiker, here's step no.1 to getting better communication with Google: get rid of Matt Cutts as the de facto communications guy of Google. Get rid of any "1 guy" being the communications person. We don't need one person. This is propaganda (sorry to "Google Bash"). It's ALWAYS propaganda when you have one person and the communication is one way. Think back to historical events. One person dictating to many. We need a dialogue. A dialogue is not one person to many, one way. It's many people to many people, two way. It's OPEN. It's free. Clearly, Google don't want to do this, so they never will. End of story. No debate. No "what ifs", no "maybes", no nothing. Google don't talk to webmasters, unless on their extremely strict terms, which are terms I personally have never found pallatable.

This isn't "Google bashing". It's reality. Might as well try to get a dialogue with the NSA about our privacy.

[edited by: ColourOfSpring at 10:19 pm (utc) on Jun 15, 2013]

[dvduval]

There is lots of helpful information here, and I disagree. People are frustrated, and when put in a corner, their frustrations come out too. I like how you are doing a "Fat Wallet" style info post at the beginning. Not everyone will even take the time to read all the ranting anyway (and surely some could be removed or moved).

[dvduval]

Maybe we should havea Google Townhall Meeting with Matt Cutts at the helm. LOL

I have a feeling Matt Cutts would not be allowed to do that because then specifics would have to be covered. Instead, general philosophical principles are used. It makes for a warm feeling. Real life examples don't work that way, and remaining esoteric is very much a style of that one-way communication.

[ColourOfSpring]

Instead, general philosophical principles are used. It makes for a warm feeling. Real life examples don't work that way, and remaining esoteric is very much a style of that one-way communication.

Expect more of this in video format:

"A Mr. M.Cutts of California asks....why does Google favour brands so much? Well, great question! Actually, you know, we don't favour brands so much as the signals they give us, and we believe those signals show that they offer better 'value-add' for our searchers, and that's why we rank them so highly, and remember folks, just keep building great content, and thanks for watching"

[edited by: ColourOfSpring at 10:38 pm (utc) on Jun 15, 2013]

[seoskunk]

Hitchiker to open a dialogue with google isn't it neccerary to understand where google are comming from ?

The fact their profits are up and share price is soaring since penguin and panda were introduced.

The fact that they are a supplier of information to the NSA.

The fact that they have gone on record denying both these published facts. As early as 2006 in the case of CIA and NSA. And mentioned previously in the thread with regard profits.

[hitchhiker]

remaining esoteric is very much a style of that one-way communication.

Precisely. They are being allowed to do a lot of this BECAUSE we haven't proactively pushed for any two-way communication.

Matt Cutts

I'm not asking that Matt Cutts do it. I believe that's part of the problem, he can only do 'so much' at once. I would like to see a lot more engineers organised and out in the wild.

I'm sitting here watching 12 years of solid work slowly roll down the toilet for no good reason.

I'd like better dialogue with the people who are responsible for curating the competitive environment I work in.

[heisje]

Some people are not comfortable with reality, facts & numbers - therefore devise utopias to sidestep them.
I have got news for you: utopias don't work.

However, nobody is going to stop you from trying.
Good luck!

.

[seoskunk]

Ok have a second question to suggest.

"if the NSA can rely on google data to protect its citizens then how come google were not able to marginalise spam search results without blanket sweeps like panda and penguin. Did google produce panda and penguin as an excuse to drive profits and already had full knowledge of who was driving web spam ?"

[TypicalSurfer]

Hitchhiker, I hear ya on what you are saying but you need to understand that the google you think you know does not exist. Larry Page does not even want to be associated with "search", answers, yes, knowledge, yes but no not search, that ship sailed.

If you create amazing content someday you may have the honor of having it incorporated into the "knowledge graph" with some attribution but that's about it as far as your utility to google is concerned. They are just not that into you, or anyone else, they can put junk in the SERPs and still make money, that is what counts.

[turbocharged]

I'm sitting here watching 12 years of solid work slowly roll down the toilet for no good reason.

Unfortunately you have a lot of company, and how these people react will be different from one another. When there are bills to pay such as mortgages, utility bills, etc., it is no laughing matter to anyone. I would expect a lot of whining, venting, emotions (or whatever anyone wants to call it) on the subject of Google because the problems webmasters are experiencing are so very widespread and impacting many sites that do abide by the rules.

It's also easy for people to get offtopic, including myself, since Google has such a wide reach that extends well beyond search. Many of these Google owned properties do however find their way into the serps such as maps, products, videos, blogs, etc.

I really don't expect Google webspam employees to spend much time answering questions on forums as that is not their core function at Google. However, it would be nice if they did create a department that could act as a liaison and answer some questions that members of the webmaster community have. To this date this responsibility has largely fallen on webspam employees or they have answered questions on their own free will (which I suspect is the case many times). Contractually Google webspam employees may be limited in what they say, where they say it, etc. which may make it difficult for them to comment even if they really wanted to.

Because so many clean websites are hurting right now, and I mean many, it is time for Google to consider allocating some funds for a department that can answer select questions from the webmaster community. Yes, Google already does this to some degree in their product forums. But to be honest, I think most people posting there with legitimate problems are turned off by the lack of moderation and hateful responses. Add moderators to my wish list for that forum too so that it serves its primary purposes - webmaster help.

I think what you wrote in your original post hitchhiker was quite thorough. I'm not here to soil your thread and certainly won't. But there are a lot of problems going on with not only webmasters, but Google as well. A lot of emotion and speculation is getting tossed around, and it's hard to control this with so many different parties hurting right now.

[ascensions]

I'm sure there will be some disagreement, but for myself, the bingiton campaign illustrates exactly how broken Panda made Google.

It's an objective census of the general public, which appears to prove Bing is better than Google... something, I would have whole heartedly disagreed with before Panda (but now I generally do).

Google can argue Panda to their blue in the face, but I've seen no evidence to suggest the search engine has improved because of the algorithm changes in the last few year.

Not to mention, while they're a highly intelligent crowd at Google, they seemed to have forgotten the psychology of what they're doing.

I'll use this analogy. A dictator puts his citizens out of work by his policies, so the people, now un-employed and starving have nothing to do with their time. They turn to crime, and corruption gets worse under that sort of system, because there is no reward for civil obedience (in our case making good websites). Eventually all these people meet in the square, and start a riot, eventually they over-throw the dictator and kill him.

The system in place can only lead to a worse web, and more corruption.

A once white-hat webmaster of two years ago, might now dabble in the more nefarious activities of the web (what Google is fighting against) because he or she has less to lose, and/or has more available time since his time isn't fully occupied by a system that benefits him/her (e.g. pre-Panda Google). Google is generating the exact type of person they're trying to fight against.

[Mac_Guy]

Since my orders from my biz are very few since all the animals updates, I have time on my hand to read all the posting here without distraction ! lol ..

Anyway, good posting and interesting opinions here.

[coachm]

Great thread. On the issue of communication, I suspect since Google hasn't communicated with web owners so far, it's not going to happen without very strong public pressure (like a light's out on Google day).

Been playing around with the idea of writing a book about Google's effects on OUR(?) Internet, since I have a good relationship with [my mainstream, but un-named publisher].

Problem with that is the research time it would take, getting numbers, sources for quotes, case studies from webmasters, etc.

I'll have to think on it.

[Awarn]

Okay this is pretty simple if you look at it. We are being outranked by garbage. Why is that? We don't know. So now it is an ethical issue or pride issue. Spam works, blackhat works, we all see it. Now the choice, let the business fail, change my lifestyle etc or do I change my beliefs? Google says they want whitehat. They also say the disavow works, there is no penalty on your site, we are fighting spam, we don't spy on you etc etc. Lets look at what actually works, doesn't matter if its white, gray or black. If Google gets too much they will fix the algorithm. These are the rules today so lets play by the rules as they are presented and not what we are told. So what do you see getting the ranking? Churn and burn? splogs? heavy keywords? Legally all these techniques are legal.

[mrguy]

With the last update, Google has made the end for small business.

People have been saying that since I started following the Google dances in the early 2000's.

Yes, in today's Google, it is much harder for a small business to succeed, however if a business is going to base their existence on free traffic from Google, then they are setting themselves up for failure from the start.