The importance of theming
I am sorry, but I think I misled some people with this statement. What I mean is the importance of Authority. You can be a web site with a reasonable PageRank and very very heavily themed to a specific subject and you will find yourself just vanishing from the SERPs as a result of this Austin update. The reason is that you are not an Authority site and there are hundreds of "Authority sites" out there that have just bumped you down the index. The important thing is you have not been banned or penalised in any way, you have just been pushed down by sites whose relevance is highly questionable.
So how do you become an Authority site?
a) a large site
b) linked to be sites themed in the same area
c) linked to by other authoritative sites with the context of the link being relevant
d) having a reasonably good PageRank (which kind of follows on from the above)
So why have sites disappeared?
We are not convined that there is a database of "trigger words" that result in a new algorithm being used. Rather we believe that there are strong themes and weak themes. A strong theme is jobs or hotels, a weak theme is accountants. The stronger the theme the more dramatic the effect on search results. In any event it is a little bit academic which of the above is correct, what is beyond doubt is that if you search for accountancy jobs in new hampshire for example the results have changed dramatically and instead of a whole load of web sites appearing (or indeed spammy pages) now you get hundred and hundreds of job portals, university sites and other authoritative sites related to the word "jobs"
And here is the Nub!
This is the single most important thing to note about the way google has changed. Do lots of searches for 3,4 or 5 word search phrases like the one above and look through the results and you will find many many examples of sites that simply are not a good match, and here is what is happening:
In the above example the strongest themed word in the search term is "jobs". So, Google collects all of its Authority sites related to that word and works out which ones are a match.
Now here is the amazing bit: as long as these authority sites contain the word "accountancy", "new" and "hampshire" they are a match, EVEN THOUGH those words may be on different parts of the page.
Get this, it is very important - if a jobs site on a particular page has one job that is accountancy jobs in los angeles and another job that is sales jobs in new hampshire, then Google says, yes that page is relevant!
You may have a bit of difficulty replicating this for job terms unless you go really specific with the discipline or area, but what is a much better example is this:
Take a specific town or village and put it at the beginning of a search query and follow it by country house hotel (for example "waterloo country house hotel" which by the way there is no such name) and just watch in disbelief as the portals all come chargiing at you!
Why does this matter? Well, we have a client who is XXX Country House Hotel and was always number one for that term as he properly should be. Now, search for XXX country House and he is number one, search for XXX country house hotel (now the uthority trigger kicks in as the word "hotel" is there) and he is not in the first 200 results. When you examine the results it is just staggering - the first two are fair enough because they are portals that sell rooms in his hotel, but from then on EVERY SINGLE ONE IS IRRELEVANT and is producing the wrong hotels. So for instance in one case it is XXX Guest House in another part of the country. Why does that site come up? Because on that page in one place there is XXX (the wrong guest house), in another part of the page there is YYY Country House.
So, Google has said you sites are all authorities on hotels and provided somewhere on one of your pages there exists XXX, and another place country and another place house, then you are a match and a match above a web site that contains the full term (being of course the hotel site)!
So what does this mean for optimisation?
What it means for web site owners is that life has suddenly got much more difficult for them, and for all those people trying to get rid of web intermediaries I am afraid that Google has just made this almost impossible.
Your average small business has for a long time accepted that they cannot realistically achieve top ranking under generic terms but this was compensated for by the fact that they could get top under niche phrases, which frankly is a bit like the real world of corporate versus smaller niche businesses. But now that opportunity has slipped for many of them.
In terms of what it means for professional optimisers, well frankly not a lot. It is just a case of moving on to the next optimisation method. Watch out Google, the number of portals you have to index is just about to go through the roof! If you thought 6 billion pages was a lot to index, we think you can add another few billion to that figure in the next few months.
What about Google?
Well, we think that for one and two word searches this has probably been a good update, although arguable nothing much has changed in the results in these cases as theming and the portal effect was already a key factor for top generic terms.
However, for 3 and 4 word terms the results are absolutely terrible. I I am looking to contact the XXX Country House Hotel I have absolutely no chance of finding them, and the portals certainly are not going to give me their number as they want to sell me a room directly.
We believe that as long as this algorithm stays google will lose market share. Already there is talk of it. We ourselves are handling traffic at the rate of about 50 million visitors per year and whilst we lost about 10% of our positions and a proportionate drop in traffic from Google, we have already seen traffic increase from other search engines. It is only a trickle now but we think it will continue.
The greatest beneficiary will probably be FAST which is quite ironic. For a search engine that could be adjudged to have been a clone of the old Google (remember when it was a good search engine back last week!) and therefore unlikely to win market share they must be rubbing their eyes in disbelief at how google has just stepped aside and let them be the only search engine left delivering relevant results!
I have a very interesting question that I wonder if anyone knows the anser to - does Google ever do usability testing of its search engine, i.e. bring in a thousand users and tell them to start searching? It would not surprise me if they did not, which when you think about it is quite staggering considering the position it is currently in and the innate value in the quality of the results.
Light at the end of the tunnel!
There is one glimmer of hope for webmasters - and that is that Google recognises how bad an algorithm this is and reverses it, or amends it so that the results are not so irrelevant for 3 and 4 word searches. However, you would not want to be banking on it.
I hope this helps clarify some people's thoughts and as I said if you want to see the full report when it is ready, drop me a sticky mail.
[edited by: Marcia at 6:22 pm (utc) on Jan. 31, 2004]
[edit reason] Pls refer to previous stickymail. [/edit]