Google vs AllTheWeb - Web Page Counters

The document count battle, something is wrong.

[cherrytron]

I read this:

"... We [Fast/ATW] believe there are approximately 30 Million crawlable servers globally, two-thirds of which have been blacklisted as spam servers." - Stephen Baker, FAST’s Director of Business Development and Marketing

Current search counts:

ATW searching 3,151,743,117 web pages
GG searching 3,307,998,701 web pages

If ATW is blocking 2/3 of all servers, wouldn't it be reasonable to assume that they would have 2/3 less documents than Google who doesn't block them?

Where is ATW finding all the other pages to make up for all the web pages they don't index because they are considered spam?

Someone is telling porky pies ..

[Brett_Tabke]

Or they both believe there are 9 billion pages on the web and 6 billion of them are spam.

[cherrytron]

but google doesnt say that it actively bans 2/3s of the web .. only ATW

so by that rational, ATW reckons there are 9 billion, of which 6 of them are rubbish ..

does this mean they are accusing Google of not being able to find the other 6 billion .. hehe ..

keep searching google .. they are out there .. you'll find them ..

i doubt it :)

[MonkeeSage]

I thought Google did at least three levels of filtering -- DMCA, spam and dupe content...is that mistaken?

Jordan

[cherrytron]

I spoke to a guy at Google on the phone, Kaiser Soze, and he said "yes, those are the 3 things we use for filtering".

"The greatest thing the devil did was convince the world he didn't exist."