Forum Moderators: bakedjake
The Times, December 23, 2006Founder of Wikipedia plans search engine to rival Google
James Doran, Tampa, Florida-Amazon.com is linked with project
-Launch scheduled for early next year
Jimmy Wales, the founder of Wikipedia, the online encyclopaedia, is set to launch an internet search engine with amazon.com that he hopes will become a rival to Google and Yahoo!..."Essentially, if you consider one of the basic tasks of a search engine, it is to make a decision: 'this page is good, this page sucks'," Mr Wales said. "Computers are notoriously bad at making such judgments, so algorithmic search has to go about it in a roundabout way.
"But we have a really great method for doing that ourselves," he added. "We just look at the page. It usually only takes a second to figure out if the page is good, so the key here is building a community of trust that can do that."
...Catching up with Google, Yahoo!, Microsoft's MSN or even smaller operators such as Ask.com will be a difficult challenge, Mr Wales conceded.
[business.timesonline.co.uk...]
[edited by: tedster at 12:08 pm (utc) on Dec. 23, 2006]
[edit reason] fair use of copyrighted material [/edit]
WikiaSearch will search among external links from Wikipedia. The bulk of the the profits from WikiaSearch will be donated to Wikipedia.
Oops! I meant WIKISearch.
WIKISearch = new project, urelated to this one, which will search among sites that are linked-out from Wikipedia.
WIKIASearch = old project, dead.
Unamed, as of yet = the project we are discussing.
It's all very confusing to say the least. There are already pundits claiming that they are now stuck with the name WikiaSearch, because it is already so out there in the press.
Most insitefull comment on their discussion board: "It needs a name."
For example, if we're talking about widgets, and there are 50,000 pages that relate directly to widgets, how often will a new site/page be created that is REALLY hand-on-heart, all-biases-aside genuinely better and more useful than say the top 100 biggest/most comprehensive/most popular/most in-depth of the 50,000 pages that have come before it?
Seems to me that there needs to be a way to establish a baseline "list of useful sites/pages" about any given niche topic, and then a separate and complementary process to elevate the relatively rare new-but-worthy site/page to be included in that list.
After all, as was already brought up in this thread, the search engines only show the first 1,000 results for a query in practice, regardless of how many results exist - and of those 1,000 results only the first few pages' worth are of any real consequence unless we want to start delving into the fractions of 1% of overall traffic that hit the deeper pages within those 1,000 results.
Here's the billion dollar question: Does a search engine USER get a better experience from seeing 78,000,000 results for a given query, or 100 (or 200 or 500 or whatever) hand-picked, relevant results, with zero spam or off-topic material?
Take a really deep breath, and step away from the webmaster aspects for a minute and say to yourself, which of these experiences would leave an average user happier, more informed, more likely to use that particular search engine again?
Does seeing "78,000,000 results" say to the average user "we're incredibly thorough, and you won't miss ANY information on this topic, no matter how insignificant" or does it say "good luck finding the needle of information you really need in this massive haystack of information we can offer you"?
One way to improve the perceived search experience could be to use existing search popularity data to help focus the manual fine-tuning effort, and gradually move down the long tail as the "big" subjects get taken care of.
First, you fine-tune the results for the top 10,000 searches (and synonyms thereof), then the next 100,000 and so on down the line. The smaller the number of searches, the less important it is to fine-tune them initially since the vast majority of people won't ever see them.
Even when people hit on "untuned" results - which would happen relatively often at first - they would also hit on tuned results even more often... so the overall impression of this hypothetical new search player might still be better than that formed from existing competitors which rely primarily (or solely) on smart algorithms.
Even if you lined up every "webmaster" in the world, plus everyone who's ever started a blog (and kept it going long enough to "care" that it got indexed by the search engines), or posted a list of URLs on a social bookmarking site, that's just a fraction of the total online population. Consumers of information outnumber regular producers of information by (probably - I don't have the exact stats) 50:1 and they outnumber commercial producers of information (those who depend on the web for income) by perhaps 200:1 or more.
So if you can come up with a way to satisfy those average users, you could theoretically be onto a winner even if you end up stepping on the toes of 95% of all webmasters out there (by snubbing their sites) in doing so.
[edited by: Edwin at 12:35 pm (utc) on Dec. 28, 2006]
You're making the mistake of thinking that because 1000 pages out of 84 million are relevant to YOUR needs, that those same 1000 pages would be relevant to ANYONE's needs.
Yes, maybe for any one individual they only need 1000 results, but they're not going to be the same 1000 results for every individual.
In fact with a term as vague as "digital cameras" there's going to be such a wide range of answers required (all those different models, shops, repair information, spare parts, technology info etc etc) that perhaps you DO need most of those 84 million pages in order to answer all the possible lines of enquiry that might be related to such a search.
A search on "digital cameras" claims 84,000,000 results. But try to look at more than 1,000 of those results. So how many pages do you actually need/use in a SE?You're making the mistake of thinking that because 1000 pages out of 84 million are relevant to YOUR needs, that those same 1000 pages would be relevant to ANYONE's needs.
Yes, maybe for any one individual they only need 1000 results, but they're not going to be the same 1000 results for every individual.
But I believe you missed my point entirely .. or actually someone previous poster's point .. about how many pages actually make up the core of the web.
Founder of Wikipedia plans search engine to rival Google
My prediction: This ain't it.
Happy New Year ;-]
....................................
Would that not be simply pointless?
It might be true today......but, would it not be pointless?
How do you index something that can grow so much faster than you?
By now Google must have concluded that it can't win the game based upon speed, and is looking for intelligence. It will lack here, as it is too young!
It seems they're looking to create a new thing where the humans helps shape the SERPs. Ever done a search and wanted a "This is junk" link next to certain results? (Maybe on all the sites above your own, but then, maybe not.)
The big challenge will be making sure the system doesn't get gamed, and certainly they're already making their plan, namely the "community of trust" they mention.
On a side note...
Does seeing "78,000,000 results" say to the average user "we're incredibly thorough..." or does it say "good luck finding the needle of information..."?
Give the average user a little more credit than that. I think they know that means, "There's tons of info on that. If you don't see what you need, be more specific."
Unleash the power of collaboration.
Then, please open source the search engine. :)
Wikimedia is great!
Another player in this space has to be a good thing if only to keep google on its toes.
It will however take a long time for it to get the sort of search data and history data that google holds so i cant see it making much impact in the early days - you only neeed to see how bad msn is to realize it takes a long time to establish a rival search engine.
Also, i dont think human input is the answer. It would be a short matter of time before its gamed by groups of individuals being paid to vote for certain sites or bash competing sites - but if they can work out a way to stop that then maybe its got potential?
Time will tell, but im certainly in favour of a fresh player entering the market - its certainly big enough for one more!
