wilderness - 11:51 pm on Sep 17, 2012 (gmt 0)
appeared since 2003 and without the common knowledge of others
People aren't letting them appear. Though there may still be a market for a "best of the rest" search engine.
Actually and likely that is not the reason at all, rather the transition of the masses to social media, and/or simply another progression of the so-called internet.
It's an appealing idea. Index only the sites whose owners don't know from blocking robots.
99.9999999% of webmasters don't give two hoots about the activities of bots on their site, nor are they even aware of the existence of raw logs.
the 0.00000001% that remain don't leave an impression influencing the activity of bots and/or harvesters.
What percentile of that 0.00000001% we here at SSID could likely fill the end of microscopic straight-pin.