|Interview: Amit Singhal, Head of Google's Core Ranking Team|
In addition to the Matt Cutts interview [webmasterworld.com], BusinessWeek interviewed several other Google engineers - including Amit Singhal who runs the core ranking team -- and that's a lot more than just the Web Spam team).
Nothing earth shaking, but I enjoyed this look at Google's internal "Replay" tool, that helps them back up their algo to see what happened a little while back:
|It was getting hard for engineers to [prove] something happened on Google. Other engineers would read their email 10 minutes later, and say, it's working fine for me. |
So they developed a tool called Replay where they can freeze time retrospectively and take a picture of it. I can say, What would have happened had I run this query at 12:49? Then we can go debug all the systems and say, Ah, that system failed.
Interesting read. One thing that caught my eye -- in answer to a question about experiments, Singhal says:
"Both using our internal raters, human beings, who say, yeah, good thing, and observing real users in a tiny fraction of our traffic, observing how they interact with something. And we have metrics, like clickthroughs."
He mentions both "internal raters, human beings" and "metrics, like clickthroughs". But I'm wondering if these are just limited to "experiments", or are they also used as part of the real rankings process.
|He mentions both "internal raters, human beings" and "metrics, like clickthroughs". But I'm wondering if these are just limited to "experiments", or are they also used as part of the real rankings process. |
I gather from Matt's interview that these are likely to be used to identify problems. Matt in fact refers to what I assume are 'clickthroughs'....
|If someone did 15 queries in a row and never clicked on the results and eventually left, that may be the sort of thing where you dig in and say, well, did we have horrible results? Were they looking for a picture and we never returned a picture? |
Thereís a lot of different ways we gather all that data to identify a problem. Once youíve identified a problem, thatís when the fun starts, because you can brainstorm a little bit.
This might be where that Replay tool comes in.
Reading both interviews together gives a good sense of the process.
Comments from various users on the BusinessWeek interviews with Singhal and Cutts highlight their appalling lack of basic knowledge about Google's ranking process.
The interviews (there are four in all) reveal very little. Google is very good at not saying anything.
They need to replay back to Friday 16th October afternoon and see what happened because something went badly wrong from that point in time. It looks very much to me like some filter was accidentally missed out or perhaps they have failed to fold in DMOZ data. Hopefully when they get back from a nice rest this weekend they will roll up their sleeves and add whatever vital ingredient they missed out.
I also noticed that from Friday October 16th from around noon rankings just changed in Google and it is hard to say what the reason is. I am hoping that Google goes back to how things were before Friday noon. During the weekend, these changes have remained. Before the changes, you could sort of see things moving in a certain direction but now it is harder to see that sort of movement.
"Not saying anything" yes, this is a business strategy or in other words "hiding facts".
|I also noticed that from Friday October 16th from around noon rankings just changed... |
Whether or not Google fizes Friday's shift by using the Replay Tool that Amit Singhal mentioned in the interview, we're not likely to know. If anyone has more observations about changes in the SERP, the best place to share them is this thread: Google Updates and SERP Changes - October 2009 [webmasterworld.com].
In addition to the article's title quote ("We Will Try Outlandish Ideas"), I also thought this comment was interesting - it seems to describe the need for Caffeine:
|Q: Udi talked about the search quality team doing more than 6,000 experiments a year. How does Google do that many in a year? |
A: You need that infrastructure where they can go from idea to data in a few hours.
Also this quote. It may just be a good PR-value statement, but it also sounds to me like Amit has the attitude that he describes:
|Q: Is there anything you do to make sure you don't get in a rut? |
A: That's the biggest dilemma. That's what I tell my team. What we may have is a very well-oiled oval wheel. And when the first round wheel comes along....
I would be lying if I said I had a magic bullet to solve that. It's the openness to innovation, the desire to dismantle status quo.
Sorry if I took the thread off at a tangent.
I was interested to read the bit about stemming and the first attempt not working. Sounded like the Florida update to me!
I wonder if this guy is responsible for all of the Google cock ups of the last few years. I gathered from what he says that they are prepared to do trial and error on an ongoing basis on the live data.
I thought that as well - and some of the live SERPs over the years certainly can be called cock-ups!
But the article kind of blurred the line between Udi Mamber testing interface layout changes (6000 experiments in a year) and algorithm updates (over 400 in a year).