---- I wonder if google feels dirty after putting that ad on their homepage
lucy24 - 10:26 pm on Aug 3, 2011 (gmt 0)
The days of free traffic may be numbered unless another startup launches a competing engine that revives what Google used to be, a pure search engine.
But if all the Big Webmasters whitelist robots, admitting only the ones from known, established search engines, then where would the competition do its crawling? A genuinely new search engine would only be able to offer a "best of the rest".