Search Engine Sizes

[Brett_Tabke]

Altavista looks like they are back in the size race. A single word keyword popular search was done on "mp3". It has to be a single kw since variations in search algos lead to major differences in phrase matching.

Google 13,500,000 Altavista 11,308,825 WiseNut 7,016,135 Lycos 6,140,575 AllTheWeb 6,033,096 Ink/Iwon 3,952,280 NorthernLight 3,023,047 Teoma 1,350,000 Excite 301,960 Ink/Anzwers 19,337

[4eyes]

My God, you are right!

Hmmm..

Checking some less competitive phrases the picture looks a little different.

diabetes
AV=1.06 million
Google=2.54 million

oysters
AV=68K
Google=198K

candles
AV=578K
Google=1.11 million

Is it possible that AV is expanding the database for areas of high traffic only?

Surely there can't be that many people going for their pay for play scheme:)

What can it all mean?

[agerhart]

I am still not seeing much traffic come from them....pretty frustrating! Especially since I dropped $200!

[NFFC]

I get 13,500,000 from Google?

[Rusky]

As do I

[Macguru]

>I get 13,500,000 from Google

So I do, when I click "search the Web" radio button.
I get a lot less in French.
What are the language preferences?

[engine]

Total NL DB size is 326,471,163 as of this time.

I wish I could get this check to work on the others.

ADDED
I get 525,205,226 on AV

[Brett_Tabke]

darn cookies. I thought that was low when I searched on Google - lang prefs.

[satanclaus]

I don't see much traffic from Alta anymore either. I'd say the database size doesn't really matter when there is nobody using your engine and your results are so bad you're losing users instead of gaining.

Guess I don't care much for Alta.

[4eyes]

I had a site at No.1 on AV for a highly competitive phrase, (not anymore, though)

I got similar traffic from a lower ranking on Lycos.

Similar pattern appearing on the 100 or so other sites we monitor - AV is no longer a major traffic provider for my customers, All UK based, and this may be a big factor.

As far as I am concerned AV is dying and no longer part of my long term strategy - especially since they dumped the regional engines.

My optimisation for AV consists of 'fire and forget' submission only.

NB
I reserve the right to withdraw these comments unreservedly, or even deny I said them, should AV ever get its act together:)

[backus]

AV has been giving me some gip, but recently, things are moving... strange but true.

[rogerd]

Despite some good rankings, AV traffic seems light compared to their historical share and other major SEs. If, however, they can get their act together with both a big database and good algo, they might start gaining share again.

It's my impression that SE market shares tend to change slowly. Even if the database stops updating or the algo starts throwing out poor results, the average surfer doesn't change SEs immediately. Over a period of 6 - 12 months, though, switching does occur. This happened to Excite who saw their strong #2 position deteriorate to also-ran status after a year or two. Not coincidentally, during this period the database was rarely updated, results were spammy, etc.

AV was my #1 choice for years, but I've switched to Google because the results are better. If I search for Acme Widgets at Google, the Acme Widgets corporate site will almost certainly be #1, while at AV it's likely to be buried. Eventually, even regular folks notice this kind of stuff.

[PageCount]

The exciting thing about these findings is that they confirm Greg Notess' mid-August check on database sizes and, more importantly, findings in respect of search hits by queries to the same databases. You can view his results at Search Engine Showdown [searchengineshowdown.com]. His annotations also make good reading.

A search on MP3 is not going to give you more than an idea of database size. For what it's worth, Greg ranks the database sizes as follows: Google, Alltheweb, Wisenut, NL, HotBot (Ink), AV, MSN (Ink), TEOMA, and DirectHit. That's AV coming in at 6.

What is exciting is the almost direct match in both your findings as far as search hits are concerned. Greg's final results rank it thus: Google, AV, WISEnut, alltheweb, NL, HotBot, MSN, TEOMA, and DirectHit. Like you, AV in at 2!

The only difference (miniscule at that) is in his placing of Northern Light before the Inktomi-Gigadoc database driven HotBot, where your use of iWon could've influenced results there. Also, Greg slots in MSN (another Inktomi user not tested here) before TEOMA. He places the other engine he tested, DirectHit, ahead of those tested by you, Excite and Anzwers (Ink-driven).

That jump by AV from sixth in terms of database size to a stunning second in terms of search hits speaks volumes. First it shows how AV, with a far smaller database than the others, maintains a steady fourth place in my list of engines (behind Google, MSN, and Yahoo).

Secondly, it says great deal about the engines at the level of the algorithm.

Quite what it says, I don't know, but it does talk to me...:) Of late, I've found Alta Vista and MSN giving far better 'broad-based' results than Google when I've indulged in similar search exercises. It appears that, while Google gives good results on searches loosely associated with my keywords and phrases, the closer the search moves to those words and phrases without matching them, Google loses the plot. When there's a match between key phrase and search, it comes back in.

MSN and Alta Vista don't lose the plot in this way. While they might be accused of delivering some irrelevant results, their adherence to broader search parameters - while the searcher homes in on the key phrase - snags sites, i.e mine, that are extremely relevant. In this webmaster's world, we call results such as this ‘a good thing’.

Whatever... these results tie in with my experience of what's been happening of late and - to a degree - make my week :)

[engine]

In light of suggestions that AV's regional db has been incorporated into one, I ran some tests and conclude that the current AV figure is 560,976,998 (up from 525million on August 28th). Although the tests are not 100% accurate, it does show a modest increase.

If this figure is correct, I find it hard to believe all the databases have been merged into one. Although, it is quite possible that the tests do not allow for regional data to be displayed at the same time.

Additionally, NL shows an increase from 326 million to 357,523,005 today, and appears to represent a greater increase in DB size than AV.