jmccormac - 3:49 pm on Jun 20, 2013 (gmt 0)
To paraphrase the immortal line from the movie "Jaws", You're going to need a bigger harddrive. :) That collection is 310TB of data.
@explorador This is what happened to Cuil: [techcrunch.com...]
@Chris13 The monitoring is only the superficial problem with Google. The real problem, for most people, has been the falling quality of SERPs.
Jimbo Wales tried to launch Searchwikia or Wikiasearch and that didn't work out either. The theory was to use the same approach as Wikipedia (free editing and contribution of expertise) to build a social media influenced search engine. Again the big problem with that wasn't the idea as such but rather the simple issue of quality control. The GIGO approach of spidering everything and hoping that the algorithms will give the data some relevance does not work for a Google alternative when there is no spidering strategy other than blind crawling. Google has more resources and people.