Page is a not externally linkable
europeforvisitors - 3:07 pm on Feb 7, 2005 (gmt 0)
Users aren't looking for "all the sites"--they're merely looking for pages that fit their search criteria. If a search on "purple widgets" returns 995 search results instead of 1,000, (or even 900 results instead of 1,000), how many users are going to know or care, as long as they find what they need in the first 10?
If enough webmasters get fed up and ban Googlebot, word will get out that Google's index is not complete, because their bot has been banned. People will go elsewhere to search, because they know Google can't list all the sites due to the ban.