AlltheWeb reaches 3 billion page index

Massive competing with Google again.

[lazerzubb]

1 - 10 of 3,048,540,854 Results for url.all:http:

Says itself, well done FAST team.

I think this will probably make google change the number on their frontpage in not to distant future.

[vitaplease]

nice one mcavic,

it really seems as if Google waited on purpose before releasing.. ;)
good to know there really are some young guys working at Google.

and then, as always, there's some left in the search for "the".

[bwelford]

I agree with you, vitaplease.

For a Google search on "the", I get a count of 3,480,000,000 billion web pages.

The same search on AllTheWeb shows 1,316,623,131 web pages.

So Google seems to be bigger than they say and AllTheWeb may well be smaller.

Barry Welford

[newsphinx]

Does the No. really count? IMO, it's the SERP quality which counts. Searchers won't even check the results on the second result page. What is the point if ATW shows 198,000 results or Google shows 285,000 results? I do believe ATW has a future. The money Overture spent on it won't be wasted. Probably its result will be incorporated with Alta vista and Inktomi. Yahoo is going to be a meta search engine! :o

[WebStart]

IF Search Results RELEVANCY COUNTS, Google is still # 1, and will be, I suspect, for some time. I just don't see All the Web as being close, or any others. But it's good to have competition there.

[Rumbas]

Overture Launches World's Largest Search Index; FAST Index of 3.2 Billion Documents Demonstrates Overture's Commitment to Industry-Leading Web Search Technologies

Press release [corporate-ir.net]

[Namaste]

I am very suspicious of the numbers released by alltheweb:
1. I doubt their index is as big, cause they don't have many pages indexed that are commonly found on Google. And we all know that Google does a through job of indexing a site.

2. Alltheweb's index is STALE. Do a search on any subject in Google and you will find pages that have been added upto 2 weeks back, even if they are deep. The same search on FAST dosen't even show pages from 2 months back.

3. Possibly they are fresh indexing some pages, but by and large their index is stale.

[mayor]

Google was like Fast not long ago ... good technology but low visibility, until Yahoo took them mainstream.

By the way, who is it that owns Fast now?

[WebStart]

Who owns FAST? Overture according to an earlier post (#35, from Rumbas). But Overture still uses Inktomi results after its paid listings when you use the Overture search engine itself. So go figure .....

[Fischerlaender]

Google was like Fast not long ago ... good technology but low visibility, until Yahoo took them mainstream.

But this was in a time when Yahoo was the dominant web brand - which now is Google. Just the bare fact, that Yahoo will now support ATW doesn't mean that ATW will rise as high as Google did then.

[Fischerlaender]

I am very suspicious of the numbers released by alltheweb: I doubt their index is as big

I just saw that ATW has now about three times more pages from my website indexed than Google has. Really surprising. Especially because most of my pages are dynamic with the bad "?" in the URL ...

[jpalmer]

Greetings and Gidday all,

if ATW is the biggest, why am I seeing included in ATW SERPs, urls that only G is constantly (and driving me cazy with!) indexing and including in their DB?

(I checked the ATW DB and they report those pages not in their index, but they gotta be pulling them from somewhere!) {:-]

Baffled.
JP

[Feebie]

I have found ATW results to be far better than Googles. The results are up to the point. I tried searching in for some of my keywords. The results were completely different than Google and IMO far better - not just because I was ranked well ;)

I wonder what is the algo that ATW uses. It gives amazingly accurate results according to their importance. The best part - CONSISTENCY!

[Fischerlaender]

There is one thing I really don't like with ATW now: Their sponsored results look nearly identical as the real results. Google is more honest in this case.

[Clark]

Google has updated their homepage:
©2003 Google - Searching 3,307,998,701 web pages

[businessweek.com...]

Forums are getting more interesting these days now that reporters have learned they are the best place to find out the latest and greatest about any topic. I've seen many stories started here. Been approached by reporters from here and I wouldn't be surprised if one or two stories came out of this very thread. Then the reporters come back here for the update ;)

Hi guys!

[gengar56]

What I'm wondering is, how did these guys get first hit in AlltheWeb for url.all:http:?

[edited by: Rumbas at 10:22 am (utc) on Aug. 30, 2003]
[edit reason] No url's please! Thanks [/edit]

[WebStart]

Let's see if I can figure this out: Overture owns FAST, and also owns Alta Vista, but Overture uses Inktomi results after its paid listings. I suppose integrating search results takes time, but ...........?

Then: Soon (if not already?) Yahoo, which was a directory, but using Google search results, will own Overture. Now -- for the future: what results are going to show on Overture, and on Yahoo? And how the heck can Yahoo make money after swallowing Overture with its recent FAST and AltaVista acquisitions -- two search engines hardly anyone uses (at least in the US)?

I think Yahoo has bought (or will buy) one great big lemon, except for Overture revenues from paid ads.

But the thing about that revenue is that probably half of that present revenue is paid for by Yahoo itself, since it is either #1 or 2, depending on who you trust for stats, for searches on the Internet, and it receives revenue from Overture for the listings on the Yahoo portal.

I suppose buying Overture is one way to "pay yourself," if you are Yahoo, but it still leaves little profit from Overture paid ads if you are already the owner of half the revenue from those.

[WebStart]

re previous post where it says:

"paid for by Yahoo itself,"

It should say

"already received by Yahoo itself from Overture."

[mincklerstraat]

I've also really been impressed with alltheweb recently. Great results, and they're really busy trawling my site. However, I don't think the number of pages in their index is so significant.

I've noticed in previous months that alltheweb has a much bigger problem with trivial duplicate pages in its index. I.e., www.mysite.com, www.mysite.com/index.html, and mysite.com will all show up in the SERPS. A good number of these 3 billion and what will be duplicate pages.

Another note: someone's been trawling one of my directories that's got way - ugly mulitple paramater url's for all its pages, and though I haven't looked though my logs carefully, I've got all reason to believe this is alltheweb (this would also be consistent with what others have reported on it). However, though probably being trawled, they aren't showing up in SERPS.

[spud01]

amznVibe
Poor little gigablast.com is still under 200 million.

gigablast looks like a nice engine, any other like it.

The submit url issimple and no fuss the way I like it =)