Forum Moderators: open
"Sorry, Google does not serve more than 1000 results for any query. (You asked for results starting from 1000.)"
I'm stuck! For this research I need to be able to get up to at least 10,000. Does anyone know of a way around this?
I know that the API also limits you to this as well.
I've got a limiter on my own search engine and it stops searching for results after it finds the first 100 most probably matches according to the layers of filters set.
I've just had a thought on this in relation to how Google holds data and why the algorithm has been so hard to crack lately. Have any of you read the latent semantic indexing papers available out there. Doesn't it talk about having groupings of databases according to their relationship with each other? I'm going to look into this a bit more and read up on LSI again.
Sorry to change the topic of this thread, but I started it so tough ...... ;) (I suppose it may still be on topic as this limiter along with an estimate of the total number of available results may point to Google's new system.) Any thoughts?
they should say "results 1-10 of 1000" rather than "results 1-10 of 1,000,000" as it just teasing us with this promise of a million results.
But there's some value in knowing how many (even if it's not exactly accurate) results G found in its database(s) that are appropriate to the search query. How else would we be able to gauge the competitiveness of a term?
I guess you just have to remember that google has a "do no evil" policy so they probably do have pages in the index beyond the 1,000, you just have to take their word for it.
More of a PR number than anything useful.
Have you as Joe Surfer, ever gone anywhere near result #1000? I'm guessing that by the time you get to result 50, you are more likely to change your search than to keep going.
There is no value to Google to serve up results beyond 1000.
I'm sure it's a huge concern to me. And Google didn't get where it is today by appealing only to the room-temperature-IQ surfing population.
If I see more than 1000 results for what I regard as a focused search, instead of looking for real results I try to see what kind of spam I'm dredging up (I figure I accidentally triggered some blasted SMC-drop-shipped klitchware promo) and I refine my search to eliminate it (or I use the directory search).
It is extremely valuable information -- certainly more so than the "I'm feeling lucky" button.
Pleeker, you say "How else would we be able to gauge the competitiveness of a term? ". I am not sure that this would be a huge concern for the average surfer to whom Google caters.
Perhaps not, but as an SEO kinda guy, part of my job is educating my clients, most of whom are "average surfers." It's nice, when they ask why their site isn't in the Top 10 for a 1- or 2-word query, to be able to point to that number and show how many pages they're up against.
Have you as Joe Surfer, ever gone anywhere near result #1000?
Its part of the speed "secret", limit the results that need to sorted to 1,000 and things stay pretty quick.
If people don't find what they are looking for in the top 100 results, for instance, they'll probably refine their query rather than going past the 1000th result.
Google tries to serve the average user, who will probably never have the patience to go past page 10 never mind page 100
... but more likely it is designed to speed the system.
There is no value to Google to serve up results beyond 1000.
Certainly no regular user would ever go near the 1000 results page or further - that wasn't my point. I'm well aware of the possible speed increases by using a results limiter, I use one in my search engine, but it doesn't know how many results there 'could have been'. Therefore I agree with Darkness:
I believe google obtains the top 1000 results using a 'basic' ranking algorithm and then applies more advanced algorithms to the top 1000 to optain more accurate results, possibly more advanced again for the top 100 or top 10. By limiting to 1000 google does not have to waste cpu time on processing the ranking of lower results.
This makes sense to me. (I take it the first 'top 1000 results' was a typo darkness)? Google gets all the possible page matches from it database(s) and then filters them.
My search engine works the same - except obviously on a much smaller scale and with a much simpler algorithm! It uses the entire database and using a pre-set list of filters lists the top however-many-I-tell-it-to. So if the search terms is 'green widgets' Google builds a database of pages on 'green widgets' and THEN runs it through the algorithm. This is why it knows how many pages there are (2,000,000 or so) but it has only assembled the top 1000 best ones out of that. Thoughts?
For instance, I search on widgets and it returns 10 million results, and none of what I'm looking at is really like the "spastic green thingamabob widgets" that I'm really looking for. However, if I key that in as a search phrase all to its own, I get maybe 3 results because my search is so specific. Seeing the total number of results that Google found helps me broaden or narrow my search as needed to help me actually find what I'm looking for.
Google doesn't need to actually display all 10 million results because very few people actually use Google that way. The whole POINT of Google is to cut out all the irrelevent stuff and aim us at something useful, so if my search turns up 10 million results I'm going to try searching for something else, not looking through a million pages to see if I find the right one.
Even most newbies use search engines this way, although the more advanced users who have been around for a while start to get really good at it. As somebody who searches this way myself (quite successfully), I take issue with the idea that Google should just display "1000 results found" if it maxes out. I think it's a real usability benefit to do it the way they do.
