Where has it all gone wrong? Are big brands over-represented on the SERPs?

[Goforit]

What is wrong with Google?

A personal search, brings up just the same big brand names or the same reviews companies. I just want to be able to buy a product from a small company at a great price, like you used to be able to and they give good service.

I don't want a big brand that don't care after they have sold you it, at a sky high price. Also I don't want a million words to read about the product, I know what I want.

The same rubbish written in a slightly different way over and over again, in the top million pages.

This is not a search engine, it is so difficult to find something nowadays.

Yet Bing, nice and easy does it, type it in and a huge amount of sites, guess what? They are exactly what I want! Yipppppeeeeeeeee. 2 mins and that's it, ordered done and dusted, next day received it.

Why is Google making it so difficult to find anything?

Crazy Crazy World!

[glakes]

I said:

only way for Google to lure small businesses into that trap is to exclude them from organic search

You said:

Can you please provide some proof or research to support that statement?

Sure.

Perform some searches using the keywords below (without quotes):

"water bottle holders"
"furnace pressure switch pricing"
"roadside service in indiana"

These are just a few random searches where Google is permitting arbitrage. I compete in an industry where most Google queries result in a minimum of two arbitrage listings and one Amazon listing that points to an irrelevant category page. The CPC on these keywords exceeds $1.50 per click to be seen on the first page, yet the irrelevant ads get first page placement and their only content is a search box or the irrelevant Amazon category. Apparently neither Google's arbitrage policy or relevancy scoring system applies to these websites. But this problem is not isolated to me. Many others have reported similar cases of arbitrage that drives up the CPC to be seen.

I can sum up what I see in Adwords as nothing less than a scam designed to inflate the CPC with useless ads. The inflated CPC is the money trap small businesses fall into, when they are forced to pay an inflated CPC just to appear in the world's most dominant search engine above useless ads that violate Google's own policies.

[netmeg]

You're kidding, right? That that is proof?

[rish3]

Someone very authoritative on the subject once said:

"we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers"

"advertising income often provides an incentive to provide poor quality search results"

Proving that sort of thing, in the age of [keyword not provided] and personalization, isn't simple. You can, however, draw a high level conclusion from the fact that ad clicks are growing at a faster pace than overall traffic. (oh and also, despite a notable shift of traffic share to lower CTR mobile devices)

Scoff all you want. Google is overmilking their cash cow, intentionally or not.

[EditorialGuy]

"we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers"

That statement was made at a time when paid results were often blended with or even substituted for organic results in some of the so-called search engines of the era. (Think GoTo or LookSmart.)

Context may not be everything, but it shouldn't be ignored.

[glakes]

You're kidding, right? That that is proof?

You questioned my opinion, and I gave you my answer. Please enlighten me as to why you believe what I have said is false by providing your own proof. Unless you are in my industry, make the goods that I produce and have to compete with arbitrage trash in paid search that inflates the CPC, you really have nothing to present that will change my opinion. I see the evidence of how Google is allowing arbitrage to inflate the CPC every day. Many others have documented the same, and it's been discussed on many blogs.

[rish3]

when paid results were often blended with or even substituted for organic results

I concede that you would have to something more subtle these days to accomplish a similar goal.

[Nutterum]

@glakes

Why not join the arbitrage trash then? Why don`t you go to the big e-commerce websites and sell your product through them? Why do you feel entitled that Google should provide for you but not for your competition? These are questions many fail to ask nor give good answers to.

As for inflated CPCs - this is a discussion for another thread. All in all the circle of positioning is valid for every industry - key leading brands are top ranks - proactive small/medium businesses are bottom of first page/top-middle second page - the rest and the above mentioned may opt for Adwords for additional traffic and conversions. In my eyes this is fair and works perfectly for the general case user, who by the way is the main target of Google and other search engines.

[Rasputin]

the whole of the section about inherent bias in the original paper is worth reading to see it in context: see Appendix A at the bottom of the document [infolab.stanford.edu ]

This includes for example:

"an airline...had placed an expensive ad, linked to the query that was its name. A better search engine would not have required this ad, and possibly resulted in the loss of the revenue from the airline to the search engine."

Searching today for British Airways, I see an ad for British Airways first and then the main British Airways site listing further down so 'despite the possible loss of revenue from the airline' is rather fogotten

Unfortunately as they knew all along, the conflicts between independent search and a profit motive are not reconcilable so:

"the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm"

[goodroi]

@Rasputin I don't understand why that is relevant to this thread. If I search for British Airways and Google shows British Airways, I do not see how that is relevant to the question of "are big brands overrepresented in Google?" I agree that Google has a conflict between perfect search results and profits but that is a different conversation. Am I misunderstanding what you posted?

Everyone please be careful to stay on topic. If you want to discuss something other than "Are big brands overrepresented in Google?" please start a new thread for that topic. If you post off topic comments, please do not be surprised when we move them to their own threads or just outright delete them. Thanks for your help in keeping the conversations focused :)

[Rasputin]

ok, sorry, point taken. Following the threads above mine I was following up on the 'where did it all go wrong?' in the thread title.

[roshaoar]

I don't think it's as simple as big brands being overrepresented.

In my opinion, big brands tend to do a better job at their websites than most small companies. They have more staff to create more content, more interesting pages with more relevant information, bigger more comprehensive sites that evolve, better graphics and bigger graphic budgets, more up to date SEO techniques and might even have more other sites to link from.

I also think that if you build a decent site then it's perfectly possible to outrank big sites. I outrank Amazon, Ebay and brands all over the place on some of my sites.

[Sathi]

I do not agree with you. Different person have different wants. so user satisfaction is very important. Google wants to show relevant and trusted results as possible.

[Nutterum]

@Sathi

As mentioned earlier trust and relevancy are subjective to the user. A search algorithm can not really distinguish between one SERP and the next. All the algo can do is calculate mathematical formulas and number sets of variables really fast! Thus why big brands are on TOP for general case user and query. The more long tail you go the more you actually see the granularity of the SERPs and that is absolutely expected and working as intended.

[unity100]

Big brands are over represented. This happened with the panda/penguin updates to 'weed out spammy sites'.

Let me give an example to illustrate - a very specific one :

back before panda/penguin, when i did a search for something like a very specific situation regarding mysql, a result from some datacenter tech's 5 page blog which was abandoned years ago would come up on top of the results.

and it would be the exact, very rare thing i was looking for.

the guy encountered exact same rare situation, wrote about it, and then forgot about that blog.

but the document was of lasting value, hence not only me, but also a lot of other people found and linked it.

then google decides to punish 'spammy' linking schemes. and give weight to 'authority' sites.

now all i get for similar searches is stackoverflow, stackexchange, and then again stackoverflow.

and the search result is remotely relevant, in most cases irrelevant.

now im having to go to stackoverflow and search from there to find anything remotely close to what i need, and it takes much more time. i can just cut the middleman and go directly to stackoverflow for most of what i need now - since ill have to spend effort to find the thing im looking for anyway.

that, and sometimes results aside from top 3-4 come totally irrelevant.

im talking from a user perspective here of course.

i would prefer the 'spammy' results google had before panda/penguin in which i could find the exact result i am searching for amidst a presence of spammy results than today's 'clean' landscape in which i am not able to find what im looking for, but the search results are from 'authority' sites.

[EditorialGuy]

Big brands are over represented. This happened with the panda/penguin updates to 'weed out spammy sites'.

That was true at one point, but I don't think it's true now. It certainly isn't true for the queries that I watch.

Mind you, there's no reason why big brands shouldn't be heavily represented for some queries. If Google is giving a boost to subject authority (as Matt Cutts promised early last year), that should benefit large niche sites and not just small ones.

[glakes]

i would prefer the 'spammy' results google had before panda/penguin in which i could find the exact result i am searching for amidst a presence of spammy results than today's 'clean' landscape in which i am not able to find what im looking for, but the search results are from 'authority' sites.

A lot of ordinary users feel this way. My mother told me she was so disgusted with Google's search results that she started to use Bing. Other family members have told me similar statements, and I know many other small business owners that are pissed off about it too.

When big brands are over-represented in search, it reduces choice for users. When choice is removed from the user, how these users spend their money is also restricted. This has forced many good small businesses to downsize, fold or even enter bankruptcy in an effort to save their businesses.

Anyone that thinks Google does not send users to big brands intentionally must be wearing blinders. After all, about 20% of Google's board of directors just happens to have worked for one of the biggest over-represented brands in Google's search results - Amazon. And when Google invests in a brand, through Google Ventures or Capital, that brand almost always rises to the top of the search results.

Shunning small businesses to increase profits may work for Google in the short-term, but it can't last. Small businesses, which employ 20 or fewer workers, made up nearly 90% of the American workforce when Google started releasing the Panda and Penguin excuses that took out many small businesses. As more small businesses fail, or are never started, Google will rely even more heavily on big brand spending to fuel their profits.

Maybe Larry Page and Sergey Brin see the writing on the wall when they recently provided notice they pan on selling $4.4 billion in shares this year. See [webmasterworld.com...] for more information.

[Nutterum]

@glakes - you do understand that businesses revolving only on Google organic in order to maintain profitability are not stable right?

The only reason I sort of dislike the big brand positioning is when I have to deal with an example query explained by @unity100 where I search for long tail specifics and I get the same old 3-4 top blogs/news websites filled with general information I already know, or worse yet - show results of niche directories that I have to once again siphon through in order to reach what I need(maybe!).

Google is less relevant nowadays and shows big concentrations of info or trust results. But that is to be expected I believe. As with everything (from sub-atomic particle physics to grand scale cosmology to finance patterns and capital flows) everything tends to concentrate on the bigger relevant stuff - even relevant data.

[glakes]

you do understand that businesses revolving only on Google organic in order to maintain profitability are not stable right?

Of course. That's why I sell product on all the major platforms and am considering the painstaking move to sell in big box stores as well. If Google went 404 this moment, it would have a minimal impact on my small business because Google has not been sending many buyers my way for a long time.

My complaint with Google is that they are using big brand bias to manipulate the market and reduce consumer choice, which impacts my sales. Thankfully I can reach enough buyers that exist outside of Google, and my business is growing. Many small businesses don't have this luxury. Regardless, no business big or small wants fewer sales because the dominant search engine is giving organic preference to a select handful of companies, ignoring their own arbitrage polices in paid search, etc. In my industry this is the trend Google created.

As I noted previously, nearly 90% of employees work for small businesses that employ 20 or fewer workers. If Google's search results mimicked reality, 90% of the serps would be 90% small businesses. I'd like to see 50%, but even 20% would be a major improvement over what Google is dishing out now.

[EditorialGuy]

Just out of curiosity, what does "overrepresented" mean in this context? Perhaps the OP really meant "overrepresented on page 1 of the SERPS"? I just did a search on a reasonably popular item, and while page 1 was dominated by big brands, I found plenty of vendors I'd never heard of on page 2 and beyond.

More to the point, who are we to tell search engines that they should ignore the preferences and buying habits of their users? For many items, "consumer choice" isn't really an issue. The more important considerations are consumer trust and experience--and, in some cases, convenience. (If John Doe orders a kitchen sink from Home Depot or a pair of trousers from Target, he'll know that he's unlikely to be cheated, and--if the item doesn't meet his needs--he can return the product easily to a store nearby with no restocking charges or runaround.)

Big brands were ascendant long before the Web came along. How many of us do the bulk of our food shopping at corner grocery stores or purchase our clothing, electronics, etc. from street stalls these days?

[unity100]

Just out of curiosity, what does "overrepresented" mean in this context? Perhaps the OP really meant "overrepresented on page 1 of the SERPS"? I just did a search on a reasonably popular item, and while page 1 was dominated by big brands, I found plenty of vendors I'd never heard of on page 2 and beyond.

even from a user's perspective, i can say that if you arent in the results in the first half of the first page, you arent in results at all.

in the specific, very technical and precise sql query related example i gave - i do not even wander to the lower half of the page because i know that they will be even less relevant.

to be honest, i dont remember when i last went to page 2 of google results.

maybe if i was shopping for something, i would go down to page 2. because in the end it is possible to find different prices and services for Product X from different sources.

but in case of information, if the first few results - which are supposedly the most relevant - provides me merely tangential matches to what im seeking, there is no point in even going down to the 2nd half of the first page.

[Dugger]

I think what really has some of us long timers perturbed is that the reason we got involved in this internet search engine game initially is because we saw the web as a great equalizer. The little guy with next to no cash could compete with the big guy who was rolling in dough. The playing field was even. The little guy could thrive and prosper.

That has all changed and the enthusiasm for the medium is waning as a result. Search engines are fast becoming the telephone directories of old where the guy with the biggest bankroll gets the best exposure.

On top of that it really is getting increasingly hard to find a variety of useful independent information quickly by using search engines. One has to be real determined because it is buried deep.

[webcentric]

"The little guy with next to no cash could compete with the big guy who was rolling in dough."

That has been EXACTLY the point in my book all along. It's pretty obvious it was never G's endgame though. The role of "the great equalizer" has shifted to social media and the question is, how long will it take before Twitter, Facebook, Pinterest, Linked-In, etc all go the way of the Google-bird. It's elementary dear Watson, they're already on their way. It's a simple strategy really. Throw some fish food in the water and wait for the fish to congregate. Then, bait a hook and drop it in.

Google has always been after the big fish and they've used the small frye (us) as bait. All is as it should be. The real question is, what are you/we gonna do about it? The great equalizer is a transient. Where will it set up home next?

[unity100]

"The little guy with next to no cash could compete with the big guy who was rolling in dough."

That has been EXACTLY the point in my book all along. It's pretty obvious it was never G's endgame though.

of course. in free market capitalism, all players must consolidate the market as much as they can. which creates a mechanic towards consolidation of all industries, all activities, all playing fields.

google has partnership and shares with many big enterprises. why should they be sending traffic to a random small business somewhere in the world. or why should they send traffic to sites which doesnt have adsense? why should another advertising company profit from the traffic they send? this would naturally make having adsense a potential ranking factor out of the 200 factors, with some justification.

many reasons like that one can be named and they would be quite valid.

in the end google has become a major megacorp whose shareholders care about maximizing their short term profit than thinking about the future of the web or equal playing fields, leave aside long term profits. you can make much more money by constantly maximizing your short term profit into the future than sitting and waiting for some long term profit expectation to unfold. which always comes with risks, btw. so rather minimize risks, maximize short term profits constantly - win!

google will devolve to the base denominator just like any other megacorp. that started when they hired schmidt as ceo, full force after they went big. schmidt is no angel, hence has no respect for 'do no evil', but even if he did have, he couldnt just go against the usual dynamic of wall street anyway - shareholders want profits, they want them now, and they want it at maximum.

[EditorialGuy]

even from a user's perspective, i can say that if you arent in the results in the first half of the first page, you arent in results at all.

OK, let's say there are 100,000 e-commerce sites competing to be in the top five for a popular widget query. 999,995 of those sellers are going to disappointed even if Google shunts Amazon, Home Depot, Target, etc. to the bottom half of the page.

What's a practical solution? (And remember: Any proposed solution has to benefit searchers, not just sellers.)

[mrengine]

OK, let's say there are 100,000 e-commerce sites competing to be in the top five for a popular widget query. 999,995 of those sellers are going to disappointed even if Google shunts Amazon, Home Depot, Target, etc. to the bottom half of the page.

What's a practical solution?

The practical solution, thankfully, is not something you have to be concerned about since you do not operate an ecommerce website. Which leads me to my next question. Why is it that you feel compelled to argue about things you are not even involved with? To me, your comments devalue the whole discussion because you operate only information websites. But maybe that is your intent - to devalue conversations that reflect negatively on Google.

Anyway, even if there are 1,000,000 websites competing for the same product, that does not make it right for Google to give three spots to Amazon, one spot to Walgreens who does not even sell the product, another spot to Sears that points to a 404 page, etc.

<snip>

[edited by: goodroi at 4:25 pm (utc) on Feb 17, 2015]
[edit reason] Let's remember to keep it polite :) [/edit]

[unity100]

What's a practical solution? (And remember: Any proposed solution has to benefit searchers, not just sellers.)

my solution is actually simple.

pagerank worked.

it was the best days of the web search, and it is what made google basically.

of course, like everything it was exploited. ANY system/method/format you come up with, WILL be exploited eventually. we humans are smart animals, after all.

constantly changing format and trying to run towards a system which will not be exploitable is outright idiotic, for there will not be such a system, ever.

so, instead of totally breaking the search by introducing 200 factors - ie like page speed -, most of which are totally irrelevant to what im searching for (doesnt matter if what specific document i search is sitting on a web page that comes up in 8 seconds), they should have fixed the problems of the pagerank (linking) system.

and that solution would be manual moderation to an extent, to weed out worst spammers and cheaters.

which is ironically what they ARE doing now in addition to the 200 criteria they introduced. which is similar to what they are doing in gmail too, probably. do you think that the stellar anti-spam performance of gmail is from well thought, well implemented algorithmic filters? surely not - they probably employ legions of people who identify and mark spam, as well as what input they get from the users themselves.

reimplement pagerank, warn/penalize those who do exorbitant linking schemes, manually weed out most aggressive spammers, let ~50-60% of our search results end up spam (a huge estimate still) but lets have rest of our search results RELEVANT, and lets get back the 1-2 exact matches we used to have at the top of search results before this panda/penguin debacle.

the only results im interested in are those results, personally. so much that with the current state of google's search, i noticed myself thinking of starting searching stuff at bing too - despite since a long time i have associated 'search' with google in my mind. that's when i realized that google's search started sucking real bad now.

[unity100]

this thread at digitalpoint is a guy's story from 1999 to 2015 in webmastering/seo/whatnot, and its very descriptive for this topic.

".........Let's pretend my niche is about horses for an example.
The first penguin update happened on what I call "black January". Overnight the traffic went from around 90K UV's per month to 1K per month. There were already about 12 other sites that completely copied my content that they were always lower ranked than mine but around. Other the years I made the mistake of trying to get work done at webmaster forums like this and the person would do the work then think "Hell, I'm making one!" and just steal all of my content. I didn't care until the penguin. Google took a pile of my keywords and boosted a ton of corporation and hollywood sites and put them on top. I mean 20s' of them. It was so stupid. Imagine this. You got a site that is a scientific compilation of all human knowledge of horses and google puts the IMDB page of MISTER ED tv show canceled in the 60's first. I'm not lying the results where that stupid.
I was layed of off my job this year as well..........."



[forums.digitalpoint.com...]

[unity100]

Also, this may explain google's drastic and dramatic changes :

[seroundtable.com...]

".....The first version of the PageRank technology was created while Larry and Sergey attended Stanford University, which owns a patent to PageRank. The PageRank patent expires in 2017. We hold a perpetual license to this patent. In October 2003, we extended our exclusivity period to this patent through 2011, at which point our license will become non-exclusive......"

Some time in 2011 Google's PageRank patent will become non-exclusive. And then in 2017 the patent will expire completely.

As many of you know, the PageRank algorithm Google uses today is completely different from what it was back when it was first designed. But the fundamentals are the same.

[glakes]

Imagine this. You got a site that is a scientific compilation of all human knowledge of horses and google puts the IMDB page of MISTER ED tv show canceled in the 60's first. I'm not lying the results where that stupid.

As I noted previously, nearly a quarter of Google's current board of directors were executives employed by Amazon at some point in the past. Amazon's CEO Bezos was an early investor in Google as well. IMDB is an Amazon owned property.

That's quite an interesting thread you linked to unity100. Unfortunately many people have followed a similar journey - owning multiple sites, eventually investing most of their time into their main passion only to have their sites shoved to the back of the serps well beyond big brands that just happen to mention the keyword on their page.

[Nutterum]

@mrengine - now, now. Play nice. We are here to discuss not bite at each other. If you are going to criticize do it constructively.

Also do not forget that the SERPS are heavily personalized nowadays. What we see on neutral SERP checkers is +/- 10% of the entire picture. Many people shopping from your online store are actually seeing you on first page. Hence the average SERP position provided by WMT.

And another point that I believe was missed here. Everybody deep down is a cynical egoist. And I say this absolutely scientifically. That being said everyone believes that they deserves better and find it easier to downplay the reality with the statement : "It's THEIR fault!" - in our case Google.

But trust me when I say and I believe many many old school SEO and markers here will back me up - if you do not have a unique business proposition you are shunned by Google. Simple as that. You may craft pristine flathead screws but in the end, millions of other small businesses do so as well and you will not get much attention. If you are crafting platinum unique best value flathead screws however and market it well via social, content and other means - you will dominate the niche for keywords like "pristine" "unique" and other and will enjoy the traffic that comes with it....for everything else - there is Amazon.

Hate it all you want but it has always been like that and deny it all you want but back in the 60s if you did not have the budget for TV and Radio spots you would put the same arguments about big brands and unfair play, the same way you do now.