Google - sophisticated or crude - or both

[nickreynolds]

Just a random thought that helps me cope with Google!

My thinking about Google is often that with their thousands of very clever engineers, their absolute market dominance, their clever marketing strategies etc that they are a very sophisticated setup. I'm sure they are. But maybe they are also a little crude as well. And maybe this helps us to understand why we are frustrated with unfair penalties, the apparent inconsistencies with Panda etc. Google tend to talk as though they are doing a very good job that they are satisfied with apart from a few minor tweaks and maybe we get a little bit sucked in by that. It might be they are not quite as sophisticated as they think they are or maybe that their implementation can be a little crude at times.

Let me use an analogy (as with any analogy it has weaknesses and you can't press it too far). Let's say Google's Index is like this massive book with articles, adverts, information, pictures, cartoons and loads of other stuff in it. The owner of the book says "I want to make this book better for users so I'm going to get rid of some stuff in there" So he decides that where an item is a duplicate from somewhere else in the book he'll get rid of it, where an item is "shallow" and has little value, he'll get rid of that too. If an item is by someone well repected ("a brand") he'll keep it. If an item has lots of cross references elswhere in the book he'll give the item more prominence, but if those cross references are from rubbish articles he'll give it less prominence.

The way I think of Google is that they are so sophisticated that they will then go through the book very carefully and using a predefined list of factors that they think makes a good or bad entry into the book, will make a careful and accurate analysis of everything in there and taking a sharp scalpel will very carefully cut out the stuff they think isn't very good.

The reality is that maybe Google are more like a very intelligent but slightly clumsy editor. When he sees something that he doesn't like, sometimes he semms to be very careful in editing it out, but sometimes he just tears out that whole page or part of a page. Sometimes that means the good stuff has got thrown out with the bad. Sometimes he may just not notice some of the rubbish as he turns two pages over at the same time! Sometimes, although he is very intelligent, his shortsightedness leads to an innacurate assessment as to what is good or bad.

This analogy would explain why for instance Panda has hit some sites hard that really shouldn't have been hit, while others of lower quality seem to have done fine. It would illustrate why some sites seem to be guilty by association (on the same isp or GA code or adsense code) but some other sites on the same isp, GA or adsense code, are unaffected - it's just that some were on the same page in the book.

However the torn out pages aren't destroyed, the editor still keeps going back to them. If they are changed he may put them back in, or he may put them back in by mistake, or he may realise that they should never have been thrown out in the first place - we'll never know!

Maybe it's not just that Panda is so sophisticated and secret that we can't understand it but that it has a degree of crudeness to it that explains its many apparent inconsistencies. And it's this crudeness that makes it hard to pin down how to succeed with Google or get out of Panda.

Finally another example of Google sophistication and crudeness. Quite some time ago I realised that I was getting a lot of local results in search. Even in a browser where I have cookies disabled - how do they do that? Very sophisticated! Or not, as the local results were consistently for a town 150 miles away. A town I've never visited in my life and never used the town name in a search. Even now, many months later Google search results give my location as a town 50 miles away! So what in one sense is sophisticated (tailoring my personal results to my geo location) is in fact rather crude as they can't get the location right (and I'm choosing not to tell them!)

Anyway - lots of weanknesses in that analogy but I'm sure there's a grain of truth in there somewhere!

[walkman]

Panda is good at penalizing sites, often innocent ones, and leaving them there, as scrapers take their place. And at promoting Google's bread and butter: brands. Looks like Google's brightest minds are at Public Relations, Adwords and UI divisions, figuring how to fool people and squeeze more and more money from site owners.

And they have a few sadists that decide when sites innocently caught in their net escape.

Bottom line, Google has fooled the world with their "do no evil" and "we're the smartest people on earth" crap. Don't buy it. No doubt sorting trillions of pages is tough, but 99% of people wouldn't mind using Bing and probably 80% DuckDuckGo which is essentially a one man team.

As far as I can tell Panda is applying certain metrics to sites and unless you fit them, you lost 60-90% of traffic. Just like that, and now it's content neutral, meaning content no longer matters once you're pandalized. What Google did is not brilliance, it's carelessness with a cold heart since your site could be misclassified, you display the content in a certain way that produces different metrics etc etc.

They increased the margin of error and 5 months later they don't care about those innocently hurt, especially since Goog's earnings went through the roof. Had their earnings suffered, as many here incorrectly predicted, Panda would have been pulled right away. Now they are not willing to mess the great racket they got. Google.com properties apparently got a 39% increase in earnings from last year , if siliconvalleywatcher.com is correct .

[tedster]

I think we're seeing something much different than that harsh assessment - and I think the title for this thread nails it - Google is currently BOTH sophisticated and crude. I don't think we can appreciate what they are doing by focusing on just this current moment.

What they are doing, in my opinion, is taking advantage of their current dominance of the search market to launch a massive and ambitious re-thinking of how to generate web search. Rather than only thinking short term, they are wiling to break a lot of eggs to get their new AI system really cooking. And it is a gamble of sorts for them.

This kind of machine learning along with automated algorithm revision has never been done before, but they have been making gestures in this direction for many years. Now, with Panda, they launched the project full-tilt.

The immediate problem (and Google's intentional "sacrifice") is that in many ways, search quality right now is suffering badly. This is because Google's baby AI system is still quite crude. However, the fact that Google is moving in this direction at all is computationally quite sophisticated. And with all their human quality raters, Google clearly knows that their search results are not what they once were.

Unfortunately for businesses who are caught in the wheels of this machine, the more they have depended on Google traffic, the more they are suffering.

[falsepositive]

Tedster, I like your assessment. And it is INDEED unfortunate. The best we can do, as webmasters, is help make it easier for this awfully crude algorithm to make the right interpretations about our site. What Panda has taught me is to use a much more discerning eye towards sites. The thing is that I can almost see why I was hit by Panda, but it becomes totally confusing when I see scrapers with the exact same content rank with much uglier formats and designs. I see spam sites, scrapers, malware and terribly useless pages outranking others, so if you look at it from that point of view, the algorithm has failed. But if I focus on my own site and think about Google's general guidelines, I know where I can improve and understand what they are looking for.

This is the reason why a lot of people are upset. I am not upset because I was hit -- I see where I could deserve it. But many of us are upset because we see worse results and truly undeserving sites take our place.

No doubt, the new algorithm has good intentions -- it's just not quite there yet.

[dickbaker]

Tedster, what you say makes perfect sense and, if it's the case, it's a good long term strategy for Google. In the short run, though, I think it's a dangerous business plan. Yes, they have an enormous lead over the rest of the SE's, and if you're right, they're gambling on the idea that they can afford to sacrifice some of that lead now to regain it and more later.

The greater risk is that they don't accomplish what they plan, and they lose even more search engine market share.

[jmccormac]

A famous quote from the movie "All The President's Men" springs to mind:
"Look, forget the myths the media's created about the White House -- the truth is, these are not very bright guys, and things got out of hand." :) Now there may be bright people in Google but Panda, judging by the huge crap it has taken on the web, seems to be some peabrained idea that has gotten out of hand.

Regards...jmcc

[Sgt_Kickaxe]

What they are doing, in my opinion, is taking advantage of their current dominance of the search market to launch a massive and ambitious re-thinking of how to generate web search.

I disagree, I think they're taking advantage of their current dominance of the search market(and more importantly the billions of dollars that position came with) and are parlaying that into EVERYTHING else they can muster. Browsers, phones, maps, covert cia operations, backroom deals with the NSA etc. They are changing their search engine to promote those OTHER interests. +1 was to promote the new social net, instant came about during wave, places was a pure profit use of their maps and a return making offline stores happier. Innovation or a big stage to promote themselves further? It's tough to tell. Remember the youtube videos on every search before they realized they couldn't monetize them well?

Google didn't invent the social network or browsers or maps or anything else I can think of at the moment but they did use the google brand name and cash to dominate with their version. That's not entirely fair, they do have some genius minds at work, but those minds are copying others and driving them into the ground (is flickr in trouble too with the latest Google picture offering? I hope not)

What Google does best is find ways of gathering data, you can't even put a +1 button on your site without loading a file from them that gives them more data. Twitter and facebook have a non javascript solution but not Google? EVERYTHING Google does is focused on gathering data, right up to the creepy line (according to their former CEO) and beyond.

We can indeed judge their search by the now and now is looking aweful (if you're not a shareholder). Perform a search for something in your home town and you see the yellow pages online(places). That's passing as "the best search engine of today" because of the dollars behind previous marketing efforts but if they were a new company launching with that now... god help them. Places should be tucked away under a 'local' button imo, I hate it because I turn to google for WEBSITES, not to know whats down the street. (hint: I know my area well without Google)

That's all bad because future marketing will not be as cheap, data gathering will not be as accepted and everyone is wise to how Google operates. I was enamored with Google but with other companies doing amazing things I'm not as quick to think Google can and will do it better. I passed completely on this latest +1 fad not because it wasn't an interesting product but because it was Google and I don't want to promote them further right now (because I'm upset with the current state of search).

Google is going to face the starbucks conundrum sooner or later, that being that EVERYTHING they touch ends up branded by their logo and the only way to expand and win back high priced coffee haters was to buy mom and pop type stores to run without the logo.

Google without Billions of dollars would be in tough right now, I hope they spend it wisely.

edit: Look to the bottom of this page, see that little +1 button? It grabbed my IP address the moment it loaded and is forever stored... that P**** me off. The moment I decided to reply to this thread and vent a little (lot) Google knew I was here and through their collective tracking efforts they know who I am without my telling them. Is THAT what we want from the best search engine? wow, I didn't know I had all that frustration with Google stored away, I hope they appreciate the rant.

[Swanson]

I think Google has lost contact with the real world over the past few years - it does pain me to say it but that is how I see it.

To me it is an issue of social responsibility.

In the "old days" it was valid to have a bunch of crazy tech guys create algos to "organise the worlds information" as a sort of way of creating a search engine that could give you answers to your queries.

But over time this has evolved with the nature of their position in the market. No longer is it about ranking sites on their links etc. it is about so much more.

We now know that Google tries to interpret the query intention and uses semantics and so many other variables along with trying to understand what sites are about and how people use them.

Now in the "non for profit" world that is fine - however now so many businesses rely on Google for income that is not compatible with the "crazy guys creating algos" ideal.

The problem is - how many of the Google Engineers currently understand the dynamics of running a business or understand business full stop.

What is the world experience of the current Google engineers in terms of e-commerce and all the issues around it.

Are they still the "out of uni" type characters that are sensational at coding but no real world experience.

I would say yes. Lets look at Mr Panda - I have looked at his resume and it is all about visualisation, big data sets, media recognition, blah blah.

He wouldn't know the issues about running an online business - in fact he is so far removed from the real world for business that it is scary that he has been unleashed on the biggest search engine in the world.

Google is a bunch of kids running the show.

Now that is scary.

[minnapple]

Has Google moved beyond fail-safe? [youtube.com...]

[edited by: minnapple at 12:13 am (utc) on Jul 20, 2011]

[Swanson]

Where is the understanding of business in the programmers of Google?

If programmers at Google are given such free reign as publicised then that is why everyone is now feeling the pain.

If you lead a business by data (which is what Panda is) - then you have lost the plot. How can you extrapolate what are good sites from reviewers scoring into a mathematical algorithm?

That is nonsense - and that is why Google is serving crap and why webmasters are lost for answers.

[Swanson]

I wanted to add that as important as I was offered a job with Microsoft when I was 25 to work on their Internet stuff (back in the old days) - yet at that time I knew nothing except how to write clever code.

Google is a bunch of youngsters writing clever code - but with zero world experience.

[walkman]

I have said it before that some Google fans, fanboys, coolaid drinkers, admirers or whatever you want to call them, will be sorely disappointed to learn that the company they so adored, now is nothing but a greedy, calculating and cutthroat monopolist. Like a kid learning that Santa Claus does no exist. I know, I've been brainwashed before.

Those "brilliant engineers" (implies others aren't as smart) have invented what lately...? Other than ripping off Bing and now Facebook, Twitter and Quora at the same time? I dare someone to tell me what groundbreaking engineering effort has Google lead on these few years and let's see how they match up against those from IBM, Apple or even the 'evil' Microsoft?

Giving immunity and moving to the top the best known brands can't be that hard, can it? Seems to me like the easiest thing actually. Easy, safe for Google and very profitable.

Google is fraud, plain and simple. Not because they do certain things, but because they lie about it with a straight(er than normal) face. Larry and Sergey were right when they refused to initially run Adwords because they saw it as an obvious bias. They were overruled and now we see what happened.

I just ordered 2 books, both from ex-Googlers and both rip apart the "overachieving engineers" that may be good at math but lack plain old common sense. They see numbers, data and think that all life is just that. That's why they don't care about innocent sites 5+ months later. Panda was a great succe$$--for them--and they have the data to prove it. Why should I believe anything different, because they say so?

[blogs.wsj.com...]

My sense of it is, it’s not because they enjoy warm and fuzzy social interaction and they think oh, this would be a really wonderful way to bring our friends together and build a social circle. They look at it and say, “the information created in social networks is extremely important and valuable.

If we don’t have access to that information, Google will be less valuable as an information source.” So, I think they take a much more calculated view of the value of the data they cannot get if they do not have a social network that is widely used. I think that scares them. I think it scares Larry – this is not based on any conversation I’ve had with him, I just want to be clear this is my interpretation — I would guess, knowing what I do about him, that the notion that a large body of very useful information would be unavailable to Google to search, to present to the world and to use for improving its own products, that I think is something that he would find unacceptable

Or the scam with the toolbar that shows the useless PR in return for tracking every page you ever visited? Same with G+, they want to link your real name with every site you visit, every word you search for or type across ALL their numerous properties. And then sell it for a profit, to "benefit the user" of course.

It's about time we see Google for what they are, give them equal treatment. It has nothing to do with individual engineers, but with the company as a whole. I'm sure most Exxon, Citibank, Philip Morris...or your local grocery store's employees are wonderful people as well. That, however, doesn't mean I am going to buy their companies' corporate lies.

[suggy]

But many of us are upset because we see worse results and truly undeserving sites take our place.

+1 ...whoops, has Google copyrighted that already ;-)

[rowtc2]

The bad thing is you can wake up and see all traffic gone. They live in a wonderful world,probably do not understand what all traffic drop means for a small individual/business.

For a manual penalty is normal, but algo changes that decrease >50% traffic for a website is not human at all, IMHO. And i am not even speaking about collateral damages.

[indyank]

They live in a wonderful world

while many know that G+ was introduced mainly for the data it gives them, they seem to be promoting it saying privacy is sound with G+ unlike FB.. that is something which they are promoting hardly...and funny indeed...

[graeme_p]

Definitely both. The algorithm is sophisticated as algorithm's go, but no algorithm is as sophisticated as a human assessment, which is what we compare it against.

@tedster, so where are Google heading? Are there any indications of what sort of AI techniques they are using?