The State of the Internet (2013) - Summarised perspectives.

[hitchhiker]

My personal observations, and summary of information taken from a number of forums, and the personal observations of 20 or so webmasters with a combined traffic of 10s of millions of visits per day.

I've tried (my best) to summarise what I've seen in relation to this, and not editoralise too much.

Disclaimer to newer folk: Take this for what it is, a set of observations - somewhat muddied by our personal experience and expectations. We do not work for Google, we only have part of the picture.

Summary

A fairly clear pattern has emerged: The fight against spam was HUGE. At some level Google failed to react to the crossfire generated, and collateral damage. It hit widely, but no one more than UGC (User Generated Content). In part made possible by the narrowed gathering of, and seeming reluctance to consider, wider-perspective webmaster feedback.

Why all this UGC?

We are normally an order of magnitude bigger than everyone else. Things hit us first and more noticeably. Our area is 'gray', both quality and otherwise. We fight SPAM at the site level and tend to see updates before others do (due to our reach and size). In some regards, we may signal what's to come for SMEs (small and medium enterprises).

UGC site owners are hurting (pretty much across the board).

Even StackOverflow seems to have been hit in 2013 (according to alexa*)

"Too many updates, they're being careless." - 500 a year according to Matt.

WebmasterWorld (alexa* again) declining since 2011 (it's safe, don't worry, we all love it here!)

Reported: Perhaps a general decline in search traffic via Google (now at #2 spot on Alexa - Facebook #1)

Quality doesn't seem to be much of a factor. Many authorities hit hard.

No clear examples (so far) of older UGC that survived.

Examples of things like 'blank' sites, double H1, '2005 black hat SEO' making it into the top 10 (a lot of this seems to be done at the bottom of the page) Read the thread, it's quite entertaining*

Bing showing very different results.

Panda provided a boost for a lot of us UGC. I saw my traffic rise through 2011-2012. (21.5 nov 12 corrected that.)

(Unsure) Custom software, or updating your look may have resulted in another inadvertant penalty (*cross-fire, sticking your neck out)

Non-UGC: Manual adjustments for keyphrases patching and semi-correcting the problem here and there.

"Hard to say where traffic drops are occuring, it's just 'everywhere'"

Links are at the centre of this storm. UGC Webmasters forced to No-Follow everything meant organic user preference from the wider audience was lost. Hurts everyone.

The google product forums have unintentionally become an abysmal way to keep us from getting answers. (Our experience as we ventured in there recently to hunt down some answers)

Brands may not be favoured, but at this level of chaos - they're more likely to survive it.

According to some people: Black hat has become non-viable. If that's true, it may have been done by over-reaching on too many signal patterns.

I've left a lot out (paid links), as I haven't personally been following it. This is just my own meandering personal perspective.

* "Cross-fire collateral damage." In a time when bullets are flying around you, keep your head down. In web terms, that would mean blending in with the crowd. "Use traditional software, don't do anything custom-made. No-Follow every link." That's not a great situation, that hurts everyone.

* Alexa works pretty well (in my experience) for large sites - those of us in the top 10,000

* Some entertainment: [webmasterworld.com...]

Fully personal opinion

I'm a white hat SEO, for 15 years. First website made in 1994. IMHO: Good content has never been hit so hard, and to this extent. Some may argue, the current state of affairs will result in the loss of smaller deserving businesses, those without the deep pockets to survive turmoil to this degree.

IMHO Focus on content, even now. Nothing else has a longer-term chance of survival.

I don't believe any of this is intentional on Google's part, I believe them when they say they don't favour brands. PPC would be a better place for brands in general (they are used to that and can afford it, everyone wins). Leave organic to a fair mix of SME and brand - then the ad space used above the SERPs would be acceptable.


Question to the community

---

Do we need, as a community, to establish better dialogue with Google? If so, a proper way to do that. To keep the signal to noise ratio down. Real Googlers participating, hundreds (not 2). I believe some inside the web-spam team might agree, please speak up.

---

[edited by: goodroi at 1:18 pm (utc) on Jun 12, 2013]
[edit reason] per author's request, added question to the community [/edit]

[hitchhiker]

None of us know him, we only know his work as Webspam's PR guy. In that regard:

I *like to think* he's on the right side of things, but his hands are tied (a wild guess, based on the fact that I once really appreciated his input, style and commentary). That said he's 'head of Webspam' so some of these decisions must be his...

If you look back a year ago, it was mainly just horribly ugly trolling. Now it's different, it's got more of a reasonable voice - and it's all aimed near him (because there's nowhere else to aim it).

All we can hope is that the screaming from SMEs forces them to reconsider the whole thing.

This feels and sounds like the precursors evident before the collapse of things like aol, gateway, netscape, palm, atari and others. That may be enough to warrant a 180'.

[Saffron]

whats the point of putting effort into websites and adding great content when the risk of getting a penalty for ......so many reasons. so google trying to make better content on the serps has had the alternative effect.

This is where I'm at. The drop in my traffic has left us close to bankruptcy. We are clinging on to our house by a thread.

I have an 11 year old site, which is a well known authority. My content is now below ehow, Wikihow and Yahoo answers, all of which don't always contain accurate and useful information. I am really struggling to understand how this is of benefit to people searching for information.

Another bugbear I have is they say that only a few pages of "thin" content is enough to knock back an entire site. My site has hundreds of articles, I think they're all good, but certainly some are better than others. So Google will penalise the good articles too? It doesn't make sense.

I was reading up on a rare form of cancer recently. There was a page written by a leading expert on it. He was a German doctor. His English wasn't 100%. Google have said they are going to penalise grammar too, so a site written by an expert may be penalised, because his English isn't 100%.

I think the results from Google are way crappier than they were in the past. I personally have started using Yahoo.

It feels like they broke the Internet and have taken many a good site down with them. My traffic has plunged 50% along with my income.

What annoys me the most is when people say "if you do the right thing by your site, Google won't hurt you". This is just not true. All my content is unique, I have never engaged in any dodgy CEO. And since this rubbish has happened, I have spent more time jumping through hoops trying to please damned Google instead of actually focusing 100% on generating more content.

Why too should I have to spend time schmoozing on Facebook, Google +, Twitter? I write content, but apparently social networks are used as cues too. So once again, I spend time faffing around on there (for no reward), when I would prefer to just write good content.

[FranticFish]

At least DUCKDUCKGO is NOT collecting user data

Not even the keyword searched on. Say goodbye to the one of the most useful stats metrics if they ever became the dominant search engine. I seriously doubt that many if any webmasters want that.

[mcneely]

@FranticFish .. Say good bye to all of the web stats? .. wtf .. What did we use before Google came along .. empty cans of tuna?

There are webstat progs at every turn .. Nothing wrong with doing a bit of work to accomplish an end me thinks .. and for the record? .. I've never used Google for stats ..

[content32]

Google have said they are going to penalise grammar too, so a site written by an expert may be penalised, because his English isn't 100%.

Oh no another bad decision from google, language adapts all the time... plus slang which a large portion of normal people use means the most part of the whole internet will be penalized lol

another thing is google is said to target tiered linking soon....so say you get a white hat link from another website if that website has bad links you will in effect be penalized even though the link is natural... or if you get a white hat link that is from a person whos got a website linking to them which has bad links your website will get blown up.

AND YET ANOTHER THING TO POINT OUT....Black hat use/insert contextual random links into spun articles to make there spam look more natural...sometime these can be big authority websites or the top competition websites.

one my blog just got a backlink from a scraper site.... a small extract of the article and then original source link t the bottom..... this blog .... well that confirms that it is natural to get bad spammy links.... but google doesnt see this.

[Saffron]

another thing is google is said to target tiered linking soon....so say you get a white hat link from another website if that website has bad links you will in effect be penalized even though the link is natural... or if you get a white hat link that is from a person whos got a website linking to them which has bad links your website will get blown up.

The linking thing pisses me off so much. Can I just state how much I detest Google? I have never, ever asked any site to link to my site. My main goal is to add interesting content. I don't go link farming or any other crap. I have no control over who links to my site. If they do, I don't ask them to. But Google in their infinite wisdom think it's okay to punish sites for bad links? I have looked in my webmaster tools and I have over 1,000 sites linking to me. I have no freaking idea who they are. So I have to waste more time appeasing the damned Google Gods checking those links or should I jsut focus on what I am meant to be doing, adding content to my site?

[edited by: goodroi at 12:00 pm (utc) on Jun 21, 2013]
[edit reason] Let's be professional and skip the foul language [/edit]

[kellyman]

Death by association

To Me Google has lost the emphasis on the results and put more emphasis on the spammers, it seems like they are all out at whatever cost to tackle the bad guy, and if you in anyway associated with that bad guy you get it too and yeah its your fault (apparently)

What Google has to consider that for many people ( I am one of those ) at the time of launching my business i had no idea how the search engine worked but i wanted to be on the front page, so i searched within their search engine and made a few calls from the sites listed by them on the first page to help promote my business. I did not pick the cheapest i picked the one that for me was the best fit because hey they were number one for their key term and they still are , but they have gone through 3 name changes but still on the first page through manipulation being a rookie at the time how would the average business user have any idea

My point being is i would bet 90% of the sites being hit are genuinely innocent they used the services of the company's being promoted at number 1 on Googles recommendation only to say 2/3 years later their tactics were wrong so now you get punished for something they did

Yes i agree they have to do something but they really should concentrate on these agency's and penalise the site with the links on opposed to the sites where the links are pointing or at least have a message in WMT to tell you of the links

[viral]

@kellyman You are exactly right and what is even more annoying is that there are some big sites out there that used to do "BAD STUFF" or at least what Google now considers "BAD STUFF".

But because they did it long ago and now have a genuine backlinks as well are dominating almost every niche. A lot of these so called "Brand Sites" used to be spammers, what this does is make it a very un-level playing field.

New sites have a much steeper hill to climb than the old sites. There are some big tech blogs out there that started by buying links and spamming but now are considered good internet citizens because they are big enough to not these bad links anymore.

[turbocharged]

<snip> at Google have succeeded in doing is make webmaster spend their time jumping through hoops & wasting time by "shmoozing" on social networks, worrying about "bad links", adding stupid authorship tools (which did absolutely squat), when I could have been creating more articles

The worst part about it is when you do finally find the time to create content, after jumping through all the hoops, it gets scraped, posted on sblogspot and ranks above the site you are working on. The algorithm is so bad that it can't or does not want to give original authors credit for their work when the copied content even hotlinks to all of the original images and other pages within the real site. What good is authorship when Google can't even identify where the original content comes from in such simple situations? I think Google can identify where the original content comes from, but they just choose to ignore it in favor of hosting the stolen content on one of their properties.

Sending DMCA takedown notices to splogspot is my chief complaint. If the content did not rank above so many of the originals, I would not waste my time with it. But it has become part of our expanded job duties which takes us away from developing client pages/sites. Unfortunately the stolen content can be scraped and posted elsewhere at a much faster rate than Google responds to the splogspot DMCA notices. Very, very frustrating. I wish that spam haven would close. If the content is not stolen and posted to the .com version of splogspot, it's on a .in splogspot subdomain or some other free splog hosting site.

To those that say these client sites must have problems with panda or penguin, because splogs outrank them for their original content, I say Google is complicit and further aggravates the problem by ignoring an original authors work. Under no circumstance should a site that is supposedly penalized, have stolen content ranked above its own. Anyway, many small/micro businesses have very weak backlink profiles because they serve a very small market. This is why they are easy targets now, in my opinion. It has nothing to do with being penalized but everything to do with domain authority and Google promoting its own products (splogspot) at the expense of victimized small/micro businesses.

[edited by: goodroi at 12:01 pm (utc) on Jun 21, 2013]
[edit reason] Cleaning up foul language in quote [/edit]

[CaptainSalad2]

>>>Google have said they are going to penalise grammar too, so a site written by an expert may be penalised, because his English isn't 100%.<<<

Perhaps this is a good thing (depending how they tune)? Personally I am seeing local sites across 30+40 local SERPS making it to the top based on tag clouds in the footer of the site "keyword ~ keyword ~ keyword", to improve keyword density?.

Curious, will this new signal apply to PPC sites or will they only care that organic sites return "grammatically correct" pages?

Same goes for site speed, mobile friendly version and everything else that is taken into account for the organics, are any of these signals taken into account for add rankings or is it SIMPLY highest bidder wins despite the quality of the website?

If they could encourage PPC sites (at the same time as organics) to clean up the grammar, load speed, mobile friendly versions and other signals with incentives like "slightly cheaper PPC", that would be very transparent seen to be cleaning up the web on the whole. I don't think anyone could really moan about universal rules that apply to all sites google show, organics and PPC?

If we are to believe the end user (searcher) is the true beneficiary of all these changes then they should be served the most user friendly PPC first aswell as organics...

[edited by: CaptainSalad2 at 11:33 am (utc) on Jun 21, 2013]

[kellyman]

@ CaptainSalad2

I hoped that panda would solves this but guess not,

If grammar was going to be in the algo its all too easy to make very minor mistakes, that does not mean that what your saying is not worthy with valid points

[goodroi]

Mods Note: Let's keep this discussion professional. Personal attacks and foul language are not allowed.

[FranticFish]

@mcneely - You replied whilst I was editing my post.

Yes, you'd have other metrics, like we used to with AWStats or Webalizer. Pages viewed, time on site. But would you want to do without the keyword or the visitor location? I wouldn't!

I don't want to know how old my visitor was, what they had for breakfast. But how they got to my site and what they were looking for is NO intrusion into privacy. DDG is an unfortunate reaction to big bizz snooping on web users.

[nick4u1]

I also wonder how this has affected business agency side? Web developers working with SEOs on small business projects. And I wonder who their clients are blaming for the mess...

[content32]

google will soon force us all to write according to their standards, so creative writing is out the window..... all creativity will no longer exist as we work to make the google monopoly richer and richer as they dictate over our lives even more...... well i guess there is one way to beat google in the end and not have to be dictated to by them....... by unplugging the internet...... but then again google vans patrolling around taking photos of your houses and you is a little annoying.

How are google getting away with all this?


i bet google would penalize shakespeare and charles dickens......

[diberry]

That said he's 'head of Webspam' so some of these decisions must be his...

You'd think, but some CEO's are a lot more hands-on and micromanaging than others. I have no idea what Page is like in that regard, but the switch to him from Schmidt may have impacted Cutts' latitude. Or not - just a possibility I keep in mind.

[Saffron]

I've read a lot of people on here saying they are relying less on getting traffic from Google and I am leaning more that way myself. Having market dominance, we've all come to rely on them for their traffic, but it has given them enormous amounts of power that I feel they have now started to abuse.

I would guess that I have around 5-10 competitors on my topic. There are many other sites, but they don't tend to rank as well. The competitors and I would usually hover somewhere in the top 10. I may rank no. 1 for a some articles, another competitor no. 1 for a few articles. There was no one dominant site. Since January, one site comes no. 1 probably 98% of the time. Not only that, they often get two or three results in the top 10 (usually top 5). So, you search for the term "types of wine glasses" (for example) and the results for one site in particular will show:

1) Types of wine glasses
2) Types of tall wine glasses

Which combined with them putting an ad at the top of the page (which the rest of us are now penalised for doing), pushes us further down the page.

Then we have the business of Google promoting Youtube clips. Yesterday I searched for a phrase relating to my topic and the first 5 or so links were all for Youtube. I think they are showing a conflict of interest when all results above the fold are for Youtube. I don't think that should be allowed to happen.

So, I don't even know if my site has hit a penalty, I think it's more a case of a) them deciding one of my competitors will rank no. 1 regardless of the content (and it's good, but mine is as good, sometimes better), b) ads above the fold & c) their pushing Youtube more and more.

Sites like mine get pushed further and further down the page and I have to deal with Yahoo answers & Wikihow now faring better, because their content is so much better (not).

Can anybody explain to me how this is making search results better?

[mcneely]

@FranticFish -- I don't need to know what my visitors had for breakfast either, but without Google, I can still get the term (words), location, device, time of day, pages visited and for how long --

There are ways around Google .. there always has been .. I would suggest hedging one's bets when it comes to Google, or any other search engine for that matter.

I'm not going to sit here and let Google, or any other search engine dictate the terms of my client interactions. If Google has a problem with me because my ads aren't the size they like, or I my words don't jive with their standards, then fine .. I'll make money anyway .. I've made money before Google came along, and I'll continue to make money long after Google's gone ..

I usually don't jump thru anyone's hoops but my own -- I'm obstinate, old, and stubborn in that regard .. I'll live with my page two listing till the cows come home if I have to .. At the end of the day, I'm ultimately responsible for my client base and customer satisfaction not Google.

[seoskunk]

Shameless promotion of my new post............ [webmasterworld.com...]

[Saffron]

I'm not going to sit here and let Google, or any other search engine dictate the terms of my client interactions. If Google has a problem with me because my ads aren't the size they like, or I my words don't jive with their standards, then fine .. I'll make money anyway .. I've made money before Google came along, and I'll continue to make money long after Google's gone ..

That's how I'm feeling right now. I have done everything I possibly can to keep Google happy and it has made no difference other than detract from my main purpose, to create good quality content. So now I will just do what I always set out to do and concentrate on keeping my visitors happy and stop worrying about what Google want.

I feel that this year I have sold my soul to Google and they still weren't happy. There comes a point where you just have to take back control. Google in my opinion are mis-using their power and a lot of people (myself included) have wasted so much time scurrying around trying to comply with their vague goalposts which they continually move.

[viral]

@Saffron I have one client who was decimated thefirst 2 pages as of January this year are all youtube videos and no the keywords don't have video in them or film or anything like that. These are product based keywords. The results look awful and the only way to appear anywhere on the first 2 pages is by using adwords.

[content32]

@Saffron I have one client who was decimated thefirst 2 pages as of January this year are all youtube videos and no the keywords don't have video in them or film or anything like that. These are product based keywords. The results look awful and the only way to appear anywhere on the first 2 pages is by using adwords.

its funny that, that google is saying they will target clusters of one site in serps .. but not there own youtube

I have one keyword result

where i had a new product.. did product page and it automatically got to page 4... then i thought id write a article about it and link to the product page thats on page 4. guess what the article took the place of the product page on page 4 and the product page has disappeared completely...... whats going on!

[FranticFish]

@ McNeely -

without Google, I can still get the term (words), location

Sure you can, but not with Duck Duck Go. That was what I was trying to say. I hate the fact that I'm increasingly using keyword referral data because increasingly people are logged into Google, so a search engine that sends no referral data is not what I want!

[Martin Ice Web]

where i had a new product.. did product page and it automatically got to page 4... then i thought id write a article about it and link to the product page thats on page 4. guess what the article took the place of the product page on page 4 and the product page has disappeared completely...... whats going on!

one of my clients site has a similar problem. I donīt mind cause his site appears within every targeted query on page 1 in one of the spots from #1 to #6. But in 80% of the cases his homepage will be shown instead of the detail pages with rewriten meta description tag . I am wondering about this cause the homepage does not even link to the detail page.

[arikgub]

Me thinking "the state of the internet (2013)" is reflected very well in the "state of webmasterworld forum (2013)".

Just look at the recent threads and compare them to the ole' good times. Recently it has all been bashing and whining, very little substance. Many prominent posters of the past, folks who really knew their stuff, do not post anymore. I don't know about you guys, but my own visits to the forum have become less and less frequent.

Don't get me wrong, I understand very well all the bashing and whining. I am bashing and whining myself. Heck, I have lost the small business I had bcoz of the Google's updates, and had to join a multi-billion corporation, pretty much like the one we are talking about.

Yet, a quick look at the WebmasterWorld is sufficient to realize that the industry has completely changed. I don't believe any constructive dialogue can be had with Google. What's their incentive to talk to you? They buried your content below ebay, yahoo answers and other junk, and they are making more money than ever. So why even waste their time on you?

Get a copy of "who moved my cheese" to get some inspiration, and go find a new cheese. That's the best you can do.

[chrisv1963]

But in 80% of the cases his homepage will be shown instead of the detail pages with rewriten meta description tag . I am wondering about this cause the homepage does not even link to the detail page.

The results look awful and the only way to appear anywhere on the first 2 pages is by using adwords.

Coincidence? I don't think so.

Google is no longer a search engine. It's and advertising and money collecting machine. On top of that they have no respect for privacy and they "legalized" hotlinking and bandwidth theft(see Google image search).

Not a pretty picture is it? Do no evil ...

[turbocharged]

where i had a new product.. did product page and it automatically got to page 4... then i thought id write a article about it and link to the product page thats on page 4. guess what the article took the place of the product page on page 4 and the product page has disappeared completely...... whats going on!

This may be symptomatic of the larger problem Google has in digesting link data. Linking to another may no longer be counted as a vote but instead likened to being stabbed by a knife. Get too many links, even just a handful, and your site may be permanently wounded.

It's pretty obvious that Google wants to move away from weighing link data so heavily to rank websites. Having a linked article replace what you already had indexed in the serps is one problem that Google faces. Another is a problem we are seeing more often, and that is scraped content outranking the original. Links are the common denominator in these cases, which leads me to believe that Google was premature in trying to dial down the importance of links as it has created a host of other problems.

Keep in mind that Google only needs to get it right a few times per query. This would explain Google's heavy reliance on using Amazon, eBay, Wikipedia, etc. as fillers for the top spots. Organic listings are beneath the fold for many queries, giving greater emphasis to paid ads. Expect this trend to continue as more users defect from a search engine that seeks to maximize the monetization of each query above delivering a diversified data set that is relevance based.

[mcneely]

I would expect that short of some ads, a search index might be unbiased in it's overall presentation of it's listings -- Google, lately, has been doing a pretty good job of advertising, or rather, selling it's own web properties to itself -- taking advantage of the fact that it has such a large user base.

Youtube or G+ or Chrome, just to name a few, dominates areas of the index where just a few short years ago, the organic listings were. This coupled with the increasingly ever present ads.

I've made no secret of the fact that Google is an adserver -- every step it's taken lately plays right into the hands of the corporate bottom line .. spammy scrapers or whatever all have a fairly short shelf life in the indexes .. I don't think that Google much cares about who or what shows in the top positions as long as it can free up the space for future ads in the process.

Google spent years purchasing it's following, and like any good corporate entity, is using that following to promote itself.

[nickreynolds]

"The state of the internet" and "The state of Google" are almost synonymous for many people, particularly here in the UK where, as stated by a previous poster, Google has 90% of market share. My opinion is that Google doesn't need to be a great search engine. It will dominate until something new comes along that does away with the need for search engine/knowledge engine. People use Google by default and Google's very clever marketing has achieved that.
I had tried to get away from using Google several times but always defaulted back - just because it was there on my machine.
When I upgraded my computer 6 months ago I decided not to use Google and to use Bing as my default search engine. I just use Google on another browser (opera) that I only use for checking web rankings.
You know what? I think you do. I haven't missed Google at all - (well I guess apart from streetview on G maps).

[hitchhiker]

There's fairly strong evidence that we need to see de-centralised (or multiple-choices in) search. Consumers are the people who will play a part in that, right now it's unlikely. However - if enough of us understand the basics of what we're talking about here - we'll slowly create that change in the masses.

99% of internet citizens do not understand what Google is, or what a SERP is. They don't know it's a bad SERP, they just keep searching until they find what they want.

Most people don't know what the difference between browser/internet/google is (because they have other jobs and focus on those!). They will change that pattern quickly if they think that's what they have to do next. eg: It just changes for them on a start page or something.

We are webmasters, experts or novices, we still understand this in a very different way from the people who need to make the change: The general public.

BTW: Check out SEOskunk's thread on P2P search if you're technically minded.

O also: P2P search would render copyright notices useless. It's not going to be pretty.

Also: [alexa.com...] - check out stackoverflow's drop, that's consistant with the general UGC landscape. FYI: Alexa's model is quite accurate for higher range sites.