wilderness - 11:51 pm on Sep 17, 2012 (gmt 0)

appeared since 2003 and without the common knowledge of others

People aren't letting them appear. Though there may still be a market for a "best of the rest" search engine.

Actually and likely that is not the reason at all, rather the transition of the masses to social media, and/or simply another progression of the so-called internet.

It's an appealing idea. Index only the sites whose owners don't know from blocking robots.

99.9999999% of webmasters don't give two hoots about the activities of bots on their site, nor are they even aware of the existence of raw logs.

the 0.00000001% that remain don't leave an impression influencing the activity of bots and/or harvesters.

What percentile of that 0.00000001% we here at SSID could likely fill the end of microscopic straight-pin.