I've wondered for a while why the major search engines just offer one algorithm and way of searching and focus so much on trying to second guess the user. Why don't instead search engines offer ALL of the following:
1) Default algorithm searching (for non-expert searchers)
2) Search by algorithms #2-50
3) Custom algo (searchers make their own algorithm)
4) Algorithmless searching of entire database (for real experts - no personalization, no second-guessing of semantics, just pure search expertise)
Basically, the idea behind this is that users follow the long tail, too. Different users with different intentions and different levels of expertise and willingness to learn could ALL be satisfied if search engines offered, like, fifty different search algos, the option to make your own algo, the option to search the database without filters, etc.
For example, you want to search for fried potato skins. You want to buy them. So you use a commercial algo that's worked for you before.
Or you want to find out why dogs pant. You don't know which algo to try. So you try all of them until you get results you like.
Or you want to find out all about Obscure Puce Widgets in Mackledemonia. No advanced algo produces good results, you've found, so you search the "algorithmless" database with expert searches that will extract from the slushpile what you need. YOU basically are responsible for your search results.
This system would be both easier to game and harder to game by spammers. It would involve a different kind of spam control - an organic one, in which searchers who WANTED to become better searchers could police their own results by custom making an algo or using an algo-less database search, and those who didn't could have them policed for them by using the ready-made algos.
Google, Bing, Yahoo, and all the old ones...they never really took the approach of making users into better searchers, but rather attempted/are attempting an artificial intelligence approach that tries to read the user's mind and "satisfy" her. They laid down the major highways, if you will, but at some point - maybe now - people will be ready for a network of little roads to their destinations, too, and the ability to navigate them at will. And I'm NOT talking social networking, but rather transforming search from a passive exercise into a creative one.
I suspect their focus on the major highways - The One Algo - is because they are targeting the largest market - the members of the older generations, many of whom are dealing with a learning curve just in getting online (my dad didn't even know how to use a mouse when he started) or perhaps are used to the old system of "one answer" for "one question." The younger generations are not much of a commercial target right now - but they should be. They were taught online navigation from childhood and a digital way of thinking - some from the cradle. They are likely to be more willing to "mash" up search to fit their needs - but perhaps "their needs" is a different concept in itself for them. Pacifying the user (what search engines do now) is a very different proposition from opening up a territory to creative exploration, which is inherently an activity (in my admittedly stodgy opinion) of the young. (That of course is a generalization, and the older members who read this board are HIGHLY likely to be explorer-minded, and to know others who are, too - but we're spehshul.)
Such an approach would not only focus on younger generations and long-tail users, but also take care of some of the issues with "search monopolies" that people have. When the capability to control the search is in the hands of the searchers as WELL as the algorithm builders, we find more good stuff. This is important, because it is OUR infrastructure that's being built, here - these are roads to OUR sites, both as searchers and as developers.
Anyway, not sure how to sum up here. I just find it depressing that Google and Bing have all that data, all those sites, and we (non-techies) can't access it in any other way than the filters they apply.