Google algo moves away from links, towards traffic patterns

[travisk]

Does anyone else think that Google's actions over the last few years indicate a gradual change in the importance of traffic patterns over inbound links?

Think about it... the Google Toolbar, Google Analytics and click monitoring on the SERPs give Google an incredible picture of where people are going, what pages they stay on, what sites they frequently return to and where they go when they leave.

We know that Google is pushing the toolbar onto consumers. They're paying Dell a billion dollars to install it onto 100 million consumer PC's. Imagine what the behavior patterns of 100 million Internet users could tell Google about a particular site's value.

What scares me is that this will push the blackhats from link spamming over to the busy spyware world. Imagine if I could pay some shady company to have the web browsers of 100,000 pc's randomly click on my #10 ranked link and stay on my site until Google decides that I should be #1. Who cares if these users buy anything on my site. I just want Google to THINK that they're using it. Will Google start bundling anti-spyware with the toolbar to stop this?

Am I on to something, or has this been going on for years?

[edited by: tedster at 8:38 pm (utc) on April 6, 2006]

[lammert]

You forgot to mention AdSense ads which also give Google information about how visitors traverse a site, and to which site they leave.

It is the next step in the battle to get relevant SERPs.

• First search engines trusted the webmasters of a site, until the webmasters started stuffing their sites full of keywords.
• Then search engines trused other webmasters. Pagerank was the ultimate way to calculate the importance of a site, until webmasters started exchanging and buying links to artificially increase their rankings.
• Now the only source left to be trusted is the visitor. To make this functional, Google first had to introduce tools which make it possible to communicate information from the visitor back to Google. And yes, webmasters will find ways to abuse this system also, although it will be more difficult than with the previous two systems.

[Oliver Henniges]

I'support your ideas. This morning I saw google advertise for the tolbar on its mainpage. I have the impression that all this big-daddy thing was about adding such traffic-based algos to the mainly pr-based core.

For me the beginning of the end of SEO insofar as we will never gain any insights into that data. So stick to googles law#1: Concentrate on the user.

[lgn1]

This will make it virtually impossible for the little guy to succeed. The top 10 results will be based on the well established sites.

Just because a site is well establishied does not mean that it provides the best price, selection or customer service.

Popularity feedback loops are dangerous things.

[Porter5Forces]

Sounds like the 3 Laws of i-Robot. The laws cannot be broken! :)

[travisk]

Spyware companies are getting sued in the United States. Some other countries do not have aggressive consumer advocates like Eliot Spitzer. That's why I threw in the "overseas" comment.

[Eazygoin]

Blackhat SEO is shady US as much as anything

Count me out!

[texasville]

I wonder til how long before congress steps in and puts a stop to "spyware" being installed at the manufacturer level.
The google toolbar is no more than that. If companies purchase pc's from dell with this feature, the opportunities for industrial espionage would be gigantic and how well can google guard such information.

Since most people find sites thru search engines, then it would be google chasing it's own search tale anyway. As for people that find sites by following links, google already uses link popularity.

[cleanup]

I really think you guys credit Google with more sophistication than they really have.

Check the facts and the track record of the last year, they seem unable to fix even the bugs in the system as is, let alone introducing more complications.

Google's algo/index has reached saturation point and now they are just running around trying to keep it afloat.

Yes, user data they have, megaloads of it, traffic paterns and who knows what else.

They may even come out with something mind blowing in the future, but first they have to fix and manage the system they have now before they can move on. IMHO.

[Tearabite]

Careful what you say Cleanup.. you may find your Adsense (if you have it) mysterioulsy banned for "invalid clicks"..

I for one, welcome our new overlords..

[travisk]

Cleanup, the PHD's at Google who create and tune the algos probably have very little to do with the everyday bugs that you are complaining about. This company employs thousands of employees. They can concentrate on more than one thing at a time.

[CainIV]

The idea sounds accurate, but to me it makes no sense as it would simply allows big sites that pay much money for Adsense and already have high listing and traffic to continue to dominate.

[annej]

There would have to be another component or new sites would't have a chance not matter how good they were. You have to be able to get up there to get the high traffic to get the high rankings to get the high traffic. And on and on.

I guess buying ads would help but how else could a new site ever be noticed?

[idolw]

links = votes
like in a democratic country. there is no better pattern. at least no one found it out, yet.

the problem is to answer the question:
"are millions of votes of anonymous people more important than few votes from important personalities?"

traffic patterns may be cool if that leads to get rid of the websites with AdSense and affiliate links which are only doorways forward (I mean user spends short time at these sites).
Unless the above sites are directories ;)

[walkman]

Traffic? This would mean that a search engine is the last to know /show that you're popular.

[travisk]

Too much oversimplification here.

Don't think "traffic". Think "user behavior". If 100 visitors search for "widgets", click the first site (which happens to be the most popular), but then return to the search page and click on the third site (less popular), but then never come back to the search page, the search engine may use that AS A FACTOR indicating that what the searchers were looking for was found on the third site, not the first.

[lammert]

With statistical analysis, you don't need sustained traffic to a site to know if it is liked by surfers or not. I see periodic posts here of people with new sandboxed sites which are visible in the Google SERPs for a few days, then disappear. After some time there is a new spike and the site disappears again. With these type of test samples, Google can make a good statistical analysis of the acceptance of a site by the visitors: do they stay on the site, or are they coming back to the SERPs to select another listing.

Maybe the sandbox is nothing more than a group of guinea-pig sites for Google's traffic pattern analysis algorithms. They are new so no visitor knows them and therefore it doesn't hurt the user experience if they are not visible. If during a test traffic spike the user acceptance is high, the site remains in the SERPs which can be confirmed by the fact that some sites don't seem to have a sandbox problem at all.

[Liane]

They're paying Dell a billion dollars to install it onto 100 million consumer PC's.

Although I know Dell has some pretty nice (and pricey) machines, I still perceive them to be (mainly) on the lower end of the consumer market. I could be entirely wrong ... but that's how I see Dell.

Will Google really get a reliable cross section of the world market from a tool bar on Dell machines? Or will they mainly get the lower end of the consumer market. In other words, are high ticket items likely to be sold to Dell computer owners?

[trinorthlighting]

Dell computers are at the top of the corporate world. Most major corporations buy from dell. Think about it, most people now days surf the net from work. So they are not only getting low end consumers, they are also getting corporate america as well......

They want their toolbar and search tools used instead of MSN's products...... They have to pay to put theirs on top of MSN's new tools that will soon be out..... Microsoft will have all their tools built in.... Google has to grasp at straws like this or Microsoft will eventually eat them up.

[econman]

Interesting thread.

One tiny bit of evidence suggesting that Google may already be using user data more than we realize:

We have a domain that we use for operational purposes. If you were to visit this domain, you would just see a splash page indicating the site is under construction. However, for various internal purposes we access this domain quite actively.

Perhaps as a result of our activity, I recently noticed the splash page has an Alexa ranking of roughly 100,000.

What is interesting is that the splash page also has a google PR of 5, despite having no inbound links that I am aware of, and despite the fact that Google, Yahoo and MSN all indicate it has zero inbound links.

Perhaps the PR 5 is due to our internal visits to the domain, using browsers that have alexa and/or google toolbars installed?

[Wlauzon]

I don't really think that any search engine, much less Google, is going to move to a patterns only algo.

I can definately see them adding it in as another factor however.

Not sure how it would work, but it would have to check at least two things:

1. How many people went to that site
2. How long did people spend on that site (or page)

Without #2, #1 is meaningless.

And I would add at least #3 - How many went back to that site within xx days.

I can see how it would improve the overall relevance, but it cannot do it all alone. All the other factors such as links will still have to be a part of the algo.

[cymbal]

Isn't tracking user behavior on an open system dangerous? They can't have enough tools in place to really know what the users intent was and where they were ultimately satisfied in their search.

In the scenario travisk laid out only the first page of the SERP would drive any information back to Google. That's more link a popularity contest for the 10, not all sites related to the query.

[econman]

User info wouldn't be the only type of data that Google would rely on -- logically, this would be added to the overall mix, or used for specific purposes.

For instance, user behavior data could be extremely useful in helping Google identify and filter low quality sites (aka "spam) -- sites filled with computer-generated pseudo text, MFA's and networks of simulated sites that are created for SEO purposes.

It's not obvious what user data google would rely on, or exactly what they would do with this data, but through statistical analysis they should be able to find all different sorts of interesting and distinctive patterns.

For instance, I would anticipate that there are some very distinctive user patterns associated with MFAs -- when people land on one of these pages from a SE, most of them will immediately do one of two things -- hit the back button to return to the SERPs, or click on one of the ads.

Since people don't spend much time lingering on an MFA site, with enough toolbar data Google should be able to detect them quite easily. And, of course, Google also has access to Adsense data for many of these sites, which would allow it to develop even more precise patterns it can use in detecting this type of site.

[dgrati]

- Read the friggin' patent.
- You cannot use google without accepting a cookie.
- Google checks if you go to a page and go back to SERPs and hit another page, and go back or stay, and so on.

Of course google looks at the traffic going to a page via it's SERPs.

But I don't believe it uses anything else other than the process above to determine relevancy.

Why? Because not everyone uses the Analytics or whatever else you mentioned. And if Google relies on half cooked sources of data it would hurt Google's objectivity.

Old story. Don't know why the big fuss.

[catch2948]

Interesting discussion ... However, there is something that I haven't seen mentioned yet. That is the value of consumer information in itself ...

I have felt for some time that Google has perfected the "art of search" about as much as it can be (artificially, that is). Blackhat SEO will never leave us. As with anything else, it just changes with time. So, it is just a game. But for Google, it is a game where they are spending billions of $$$ to try to win. In my view, Google as a company, is consistently working to move away from this “game” by constantly diversifying itself.

Think about something for a moment. What is the purpose of running a search engine?

Unless you can figure out a way to get visitors to pay you for the privilege of using your search database, you can't make any money from them. So, in my view, this leaves only one other profitable alternative, advertising. Whether you decide to watch what people are looking for, and start offering products and services to meet those visitor needs, or sell advertising to those who wish to, it's all about advertising.

So, now we have a way to make money from the search engine. We can charge a premium for advertisers to advertise in our search engine. So what’s next? Well, it would only stand to reason that if you could give more information to advertisers about the habits of those whom they wish to target, you could charge them even more.

But let’s not stop there.

Let’s say that you are the one running the show at Google. With all of the information that you now have access to, concerning what people are looking for, and how they are looking for it, what would you do with that information? Would you sit back and be content with taking the advertising revenue? Or if you saw a big trend coming, would you invest company assets, in order to “cash in” on that trend? And we haven’t even began to talk about utilizing the ability to create trends.

The real point here is not which way you choose to go. It’s more about having the options to choose from. The wealth of consumer information at Google’s disposal allows this, and I believe that they will continue to look for ways to both expand the scope, and refine the quality of this information. Please note that all of this avoids the contentious issue of actually selling the information of any of Google’s visitors.

Having a decent search algorithm and database is just a means to an end. At some point, as with any other business, what happens if it becomes more expensive to run the search engine than it is worth?

[Brett_Tabke]

> has this been going on for years?

Two Words:

Florida Update.

[b0rdslide]

We have a domain that we use for operational purposes. If you were to visit this domain, you would just see a splash page indicating the site is under construction. However, for various internal purposes we access this domain quite actively.

Perhaps as a result of our activity, I recently noticed the splash page has an Alexa ranking of roughly 100,000.

What is interesting is that the splash page also has a google PR of 5, despite having no inbound links that I am aware of, and despite the fact that Google, Yahoo and MSN all indicate it has zero inbound links.

Is this a subdomain by any chance? I've frequently found that subdomains almost automatically inherit some of the parent domains PR. As for the Alexa rankings just treat it as a load of rubbish, just one or two people developing a site and using the FireFox Search Status extension of the alexa toolbar can push the site into the top 100,000.

[soapystar]

perhaps traffic patterns can explain something that baffles me....for years we have been pushed the line that crosslinking is a no-no....

with big daddy i see this is a must-must..

for example the 'lastminute' guys.....every single page cross links with their network and big daddy has seen them massively improve for deep searches....it seems to be no problem any more to cross link if you are a big branded site..see this more and more...

[econman]

WRT the example mentioned earlier, our activity occurs on subdomains, but the PR is visible on the main domain (which is just a splash page).

For example, we use

CMSworkarea.example.com/examplepage/section/

for internal purposes related to development of other sites. If you visit

example.com

you see a simple splash page saying that example.com is parked, or under construction, or whatever.

There are no links going to example.com or any of its subdomains, yet the Google toolbar shows it as PR 5.

[FourDegreez]

econman said what I was going to say. The most obvious contribution the Google toolbar could make would not be to add points to a score, but to subtract them. If the only thing the toolbar contributed was to help filter out the worst sites from the SERPs, then that would be great.