kaled - 12:28 pm on Mar 18, 2004 (gmt 0)

It has been suggested that Google will henceforth be making small algo changes several times a month in future. Why would they do this - well here's the likely explanation.

When I was university many moons ago, people were beginning to experiment with learning algos. I did not ever get into this, but I think I can still remember the basics - it goes something like this.

1) You have a complex system that cannot be easily analysed mathematically for optimum performance.
2) You have a number of parameters you can change on the system - we'll call them levers.
3) You choose a number of benchmark tests to decide if changes to the levers are good or bad. Typically, the number of benchmarks should not exceed the number of levers but that's another story.
4) You design an automated algo that randomly moves the levers (in small increments, one at a time) and measures the results. If there is an improvement, the change is kept otherwise it is discarded.
5) You seed the algo by choosing known sensible positions for the levers.
6) You hit the GO DO IT button.

Almost certainly, Google have such a program. However, I rather suspect that they don't fully understand the limitations.

1) Filters (i.e. system behaviour) must be smooth otherwise results between the benchmarks cannot be reasonably interpolated. In other words, the whole concept can become complete garbage.
2) If your benchmarks are poorly chosen, results will be poor.
3) When a system has many levers, you have to periodically throw in some wild changes, otherwise the program can become trapped in peaks or troughs.

In the past, I'm sure Google has done all this in private and the end results simply appeared at the monthly dance. However, maybe Google are so confident of their system that they now plan to go public with this. On the face of it, that might seem stupid, but if a decision has been taken to measure user interaction and base benchmarks on that, then they have no choice. The simplest measure that they might use is to track the number of results that a user clicks for a search, together with the position of those results in the SERPS.

Kaled.