Googlebot is a phenomenal crawler. The best at what it does and I can't see it beaten anytime soon. However I can't help wondering if in order to create a fair market that Google should be forced to become a raw data provider and a market of Search Engines would use that data to produce their own search results. Of course they would be able to choose different providers of data to enhance their results and Google Search would become one interpretation of googlebots raw data alongside competing Search Engines that may choose to interpret the data another way. This could create a fair and competitive market in Search.