Possible Biases in Google's Search Algorithm

[aristotle]

It seems fairly obvious that Google's engineers can tweak the search algorithm to favor certain types of sites, and also to be biased against other types of sites. For example, adjustments have apparently been made to give an extra rankings boost to well-known brands. Another possible example is the Panda update, which is said to have been directed against large "content farms".

I've been wondering about other possible examples, such as small sites vs. large sites, new sites vs. old sites, ad-free sites, etc.

Does anyone have any opinions about this?

[Shatner]

Big corporation: Google will let you do whatever you want and always give you priority in rankings.

Small business: Google will penalize you heavily for things you don't know about and never tell you why.

[Bewenched]

From what I'm seeing sites like ebay and amazon, which have all sorts of "Doorway pages" on other sites are not being penalized and they should be.

Now either money is changing hands under the table or google is allowing these conglomerate monster sites get away with murder.

[netmeg]

I have nothing but small businesses that didn't get socked. I don't do any large business clients at the moment. (And at least two of them compete with Amazon selling competitive products)

However, my clients all *do* have brands of one sort or another.

[browsee]

Interesting Topic. Just saw one example of Google bias. Paul Edmondson,CEO of HubPages published one article on Hub Pages. Later same article was published in Google Knol, and Knol was on the first page. This is exactly what scrappers are doing and topping the search results.

[tedster]

When you say it "was on the first page" - for what search? For the exact title, for a long quote, or for an important keyword phrase that the original used to rank for?

As bad as the scraped content situation can be, I wonder if most of the examples we are discussing are not actually displacing the original for a real traffic driving query. In other words, Google is focusing only on high traffic searches and not currently addressing the exact phrases very well.

Even before Panda, it was relatively common for a page not to rank at the top, even for the exact title. Some other factor seems to be displacing text-match even further these days.

[browsee]

Here is the link to Paul's blog about "Frustrated by Google Panda Update"

[trainthoughts.com...]

BTW, I don't support HubPages, I believe that they deserve a demotion. But, original article should be in the first place.

[tedster]

That is a case that supports my point - the query where the knol outranks the hub is not a realistic search, it's a 28 word exact sentence in quotes. In other words, why should Google even care about getting that result "right"? Actually, on an exact title search (no quotes) Amazon outranks both of them!

I'm not bringing this up to defend Google - I'm bringing this up because I believe the situation can tech us something about the way the algorithm is actually doing things right now. Search has not been a pure text-match document retrieval system for a long time.

[indigojo]

If anything the 'biggest bias' now is the lack of quality commercial results when looking for actual products or services. The likes of Alibaba and Amazon do fine but when you look for anything remotely specialist the only thing relevant on page 1, 2 or 3 are Google Adwords adverts.

The results are now littered with educational journal papers in PDF format and patents plus a plethora of other useless junk only relevant to the 'high brow' scientist NOT the general consumer trying to find, investigate and buy stuff.

I am now personally having to go to Bing for the first time in my life to get this info - rather being knee deep, page 12, page 13 plus of Google's SERP's. It wouldn't be a daft proposition to suggest that a commercial company with shareholders needed to bring the value of it's own clicks back into the fold rather than paying 70% to content/Adsense affiliates, one simple way - bury that information deep in the index - a hidden agenda or just something that the Google academics just don't see in all their wisdom?

And another winge, does Google not want the world to have websites anymore that it doesn't own [maps.google.com...] Google Local “Business Photos”

[indyank]

Nope, Knol is no.2 for the article title that Paul Edmondson is referring to in his blog post.

Tedster, I do agree with you on the facts that you had mentioned.

As bad as the scraped content situation can be, I wonder if most of the examples we are discussing are not actually displacing the original for a real traffic driving query. In other words, Google is focusing only on high traffic searches and not currently addressing the exact phrases very well.

I am not sure how it was before panda for exact matches of titles or any sentences in articles.

Post-panda, the originals are lower than the copied stuff for those searches.But you are right that they don't displace the originals for the keyword searches.The copycats are not found around, though the original goes down in rankings for such traffic driving keywords.My feeling is that the copycats are hurting the originals for such keywords.

But these should affect sites like wikipedia as they too are copied by others.Is google maintaining the index of "article originators" for a select sites and ignoring it for others? Is this how they are able to get back sites out of panda?

_{update: Yes amazon is no.1 and knol is actually no.2}

[edited by: indyank at 3:44 am (utc) on May 6, 2011]

[Kenneth2]

I'm one of the victims of google Panda and I can definitely feel the bias with the latest update

Example [a key search term for us] shows 1 site from blogspot.com & another 1 from blogger.com with copied description and cheesy layout at page 1 whereas our site is found on distant location. This is truly disheartening.

[edited by: tedster at 4:06 am (utc) on May 6, 2011]
[edit reason] sorry, no specifics [/edit]

[indyank]

. In other words, why should Google even care about getting that result "right"? Actually, on an exact title search (no quotes) Amazon outranks both of them!

I'm not bringing this up to defend Google - I'm bringing this up because I believe the situation can tech us something about the way the algorithm is actually doing things right now. Search has not been a pure text-match document retrieval system for a long time.

Was't google implementing something to deal with duplicate content just before panda? Did they quickly decide to completely ignore it?

I do agree that search is no more a a pure text-match document retrieval system.

In the stated example, the entire content is the same in Knol and the blog.Amazon has different content and its ranking above these two isn't being questioned.But the stuff on knol ranking above the blog post is the mystery.

The example provided by Kenneth is a good one.Obviously it brings some traffic to him and google is ranking the blogspot posts above the originator which is hurting his site.

Why are domains like amazon, blogspot, knol, yahoo answers, youtube, wikipedia, etc. etc seem to be unaffected by what the copycats do?

[edited by: indyank at 4:09 am (utc) on May 6, 2011]

[dibbern2]

It makes absolutly no sense to take all the components of Panda and past updates, and distill them down to "google hates small businesses".

It is a conspiracy proposition that tries to explain away technical issues in the evolving process to categorize the web with a cop out which gives the G-bot a fictional personality.

In ain't so.

[indyank]

It is a conspiracy proposition that tries to explain away technical issues in the evolving process to categorize the web

dibbern2, You seem to be having a fairly good understanding of it now. What are the technical issues that you see?

[browsee]

But the ranking for the stuff on knol ranking above the blog post is the mystery.

That's my point

[Planet13]

Small business: Google will penalize you heavily for things you don't know about and never tell you why.

I am not hear to gloat, but as a mom and pop business with me being the web developer, marketing manager, accountant, HR director, etc., I can say that we certainly fit into the role of small business.

It looks like, if anything, my two sites have seen increased traffic after Panda (knock on wood). This despite the fact that just prior to and immediately after the Panda updates, I had some very basic technical problems that were hampering my site (e.g., I accidentally noindexed my homepage, had canonical tags that resulted in 404 errors, did 301 redirects to nonexistent pages, etc.,).

And by the way, my sites have measly page rank, so not having tons of links is not a major obstacle.

If anything the 'biggest bias' now is the lack of quality commercial results when looking for actual products or services.

I don't know if that is related to Panda. There seems to have been a significant update in October of 2010 (several people on this forum mentioned major rankings drops around that time). to the best of my memory, this coincides with some statements by Matt Cutts that the main complaint that users were giving google was that there were too many commercial sites on the first page in the search results.

He kind of dismissed the idea that scraping was a big problem in the search results. Which seemed pretty strange because then in January there was the scraper update.

I think we should look back to the MayDay update or the October update update and see if possibly the decrease in commercial sites in the SERPs may have started there.

[indyank]

dibbern2, did you mean the technical issues with the site or with google when you said "explain away technical issues in the evolving process to categorize the web"?

[Shatner]

>>It makes absolutly no sense to take all the components of Panda and past updates, and distill them down to "google hates small businesses".

Fact: Big business are treated differently by Google than small business. Big businesses are not penalized for doing the exact same thing as small business.

That's is a CLEAR bias. Whether it's well intentioned or not, whether there's a good reason for it or not, that exists.

Amazon can probably do just about anything it wants and it will never be Pandalized. Never.

That is a clear bias in favor of big business over small business. Two sets of standards.

[Shaddows]

I've been wondering about other possible examples, such as small sites vs. large sites, new sites vs. old sites, ad-free sites, etc.

I started a thread [webmasterworld.com] a while back about the fact that some practices become acceptable as a site matures. Funnily enough it was recently active again.

I think it is a really important question that needs closer examination- not least because blanket advice is usually wrong, depending on the context.

Now either money is changing hands under the table or google is allowing these conglomerate monster sites get away with murder.

Or... these monsters have established a brand, which has sufficient "plus points" to withstand some negative practices. Simply put, removing them from SERPs would indicate that "Google is broken" to ordinary searches, who couldn't give a monkeys about SEO techniques, quality guidelines, or standard practices.

October update update and see if possibly the decrease in commercial sites

Around that time, we (an ecom) were given rotating traffic sets by Google- as set out in the Traffic Shaping Thread [webmasterworld.com]. I think at the time people were a bit sceptical about profiling traffic, but hopefully Brett's recent thread has put that to bed.

Anyway, we were 'tested' and on 12th October, we got a huge spike in traffic which we still enjoy today. Our rankings were subtly different, but a lot of our competitors disappeared. I tend to think that was the ecom equivalent to Panda.

Actually, I'd be pretty interested to see how many "warehousing" ecoms were affected by Panda (as destinct from those on the <drop-ship>-to-<affiliate> spectrum.

[Shaddows]

Fact: Big business are treated differently by Google than small business. Big businesses are not penalized for doing the exact same thing as small business... That is a clear bias in favor of big business over small business. Two sets of standards.

No, its one standard, but that standard takes account of many factors, one of which is how 'expected' a site is in SERPs.

You are trying to view this from a bottom-up perspective- "If I do this, and they do that, results should be the same"

Google tries to analyse who 'should' be top, then formulate an algo that creates a similar result heuristically. That often means ignoring some behaviour in one case, but taking it into account in another.

That's why Mr Panda's paper on decision tree analysis was worth a look.

[Planet13]

Regarding Big Brands: tedster alluded to this in another thread, but...

When people go to google and type in the URL amazon.com (or www.amazon.com) in to the SEARCHBOX (as opposed to directly into the URL bar), that it probably has some effect on how well they rank, and how much trust they get.

So, yes, branding is probably helpful to search results.

[TheMadScientist]

Should we have another thread titled: "Possible Biases in Webmaster's Review of Google's Search Algorithm" to be fair? I think sometimes we're a bit overly critical when it comes to their algo, and at times is seems 'negatives' are posted when the reality is they don't really apply to most searches, like the 28 word exact match search being off ... Do you really think the people at Google or the average searcher cares all that much? I don't even care and I'm not either of those.

If you think there's a 'big brand' bias, have you ever tried building a site like the 'big brands' build to see what happens? My guess is it has much more to do with the site than the 'brand name', and there's only 10 results in the top 10, so there has to be a bias, otherwise every page possible for the resultset would have to be cycled through, randomly, so of course there's a bias, just as there is on any site that has links which are not displayed in a totally random manner.

There are biases right here on this site:
Threads with newer posts are pushed to the top of the Specific Forum index page, and threads with a 'higher degree of perceived importance to a large number of users' are promoted to the home page ... Should we be upset not all threads we start are listed on the home page or do they have the right to decide which threads are and which aren't?

[TheMadScientist]

A little bit more:

1.) How many times in History do you think some of these queries people are using to point out how 'broken' Google's algo is will actually be used by a searcher? Really, stop and think for a minute.

2.) If you don't know who the originator of the content is, how bad are the results really, except they don't include your site? Seriously, from the perspective of a searcher who only wants information, how bad are the results really?

3.) They serve results for somewhere near 1 Billion queries a day, and even a 90% success rate means they Miss on 100 Million queries a day ... So you found one ... That's easy to do, now compare that to how many they get right...

Are we really sure the 'Google's Broken' threads aren't where the real bias is?

Sorry, but the whole 'broken' thing got old a while ago ... Different? Yes .... Broken? I don't think so ... Lil Baby Machine Learning Algo that's Only a Couple Months Old? Likely ... Does it Need Return Correct Results 100% of the Time to be Working and a Long-Term Improvement? Nope.

[netmeg]

+1 to TMS (you must be about +50 by now)

[Sgt_Kickaxe]

This goes BEYOND existing content.

Many product based keywords are no longer attainable. In fact if you target such keywords with an exact match title, or with exact match anchor text, your article will not rank at all. If you add a single word, such as "vintage" in front of "blue widgets" for example it becomes very apparent that a lock is on for "blue widgets".

Perhaps Google has identified product names as being targets of spam and is treating them very differently now.
Perhaps Google has identified visitors likely to buy a product and is diverting them with custom results.

The list of possibilities is long but there is a change to EXACT MATCH product keywords.

[tedster]

there is a change to EXACT MATCH product keywords

I've been hearing some reports that exact-match domain names are losing some of their edge, too. That's a tweak Matt Cutts promised to look into - maybe it's all one piece?