| This 71 message thread spans 3 pages: 71 (  2 3 ) > > || |
|Why doesn't a new and improved competitor to DMOZ arise?|
If DMOZ is so valuable but so bad why hasn't a new competitor arisen?
Do you people think that it is possible for someone with a bright idea to beat dmoz?
Hello chandubhai and welcome to the Directory Forum.
You likely have noticed that I have significantly changed the title of your post. I hope you don't mind. I prefer to have a dialogue about DMOZ/ODP that is cast, in most circumstances, in positive proactive terms: "What to do?" not "Why is it so bad?"
In your case I think you raised an interesting issue for discussion but cast it in terms that were likely to bring out more flamming posts than anything else.
It is a very interesting question though.
IF DMOZ is "so bad" then why is it that no other attempt to supplant it has ever taken hold in the past ~5+ years that it has been the de facto Google approved "directory of authority".
If it's so bad then why hasn't there been a ground swell movement to build a better mouse trap, so to speak?
Is it in the nature of whiners, whingers or complainers to go no further than to criticize? If it's so easy to "do it better" then why aren't they organizing themselves to do it better?
If so many folks are so displeased what's stopping them from doing DMOZ better than DMOZ?
Why isn't there a better version of DMOZ if it's so bad the way it is?
The ODP isn't a "bright idea," and no bright idea will ever compete with it. (Ideas are a dime a gross these days: the internet has depressed that market seriously, since all the world's ideas are available for free at your fingertips. And replicated work is somewhat worse than worthless. Unique effort is all that matters.)
It's a socio-economic system which is merely one of many IMPLEMENTATIONS of the "obvious" idea of "using the web to facilitate cooperative work on a very large project." (It isn't even the only cooperative web indexing project. It's merely the only one remaining.)
The question is, suppose someone WANTED to cooperate on an ODP-like project. (Clearly, not everyone does.) Would he be better off by making the ODP better, or starting to build a similar project from scratch? Well, obviously, making the ODP better would be a better return on investment UNLESS:
(1) the ODP tools were so flawed as to impose significant inefficiencies in directory building (which is simply not the case: the tools are a joy to use, fast, efficient, and intuitive.)
(2) the ODP community is, for some reason or another, excluding enough likeminded people capable of building their own competing community. Now the ODP certainly excludes a lot of people, some of whom are certainly interested in building a directory, some of which are certainly capable of building a directory, and some of which are certainly capable of building a community.
The questions then are:
(a) The social question: is the intersection of those three groups large enough to replicate (more or less) what the ODP community has already done, in order to begin building the "better" part? (I suspect not.)
(b) The economic question: given the niche market of general-purpose web directories, would those people choose to to go through the effort of go do that, rather than doing something else that hadn't already been done? (Again, I suspect not.)
The information system economy tends to support no more than three visible players in each market: the big commercial effort, one big public effort, and a small commercial competitor.
Thus in operating systems: MS-Wind, Linux, Apple
In browsers: MS-Exploder, Mozilla/Firefox, Opera
In directories: Yahoo, ODP, MSN
In personal finance: Quickbooks, (name escapes me at the moment), MSN Money
In word processors: MS-*urd, OpenOffice, WordPerfect
and so on.
I suspect the reason for this is: by nature, it's hard to motivate people to cooperate in competing with a cooperative project -- they either focus on competition (and thus can't work with each other) or on cooperation (in which case they just cooperate with the other cooperators.) And I suspect (hope) that most of the cooperative, knowledgeable people that the ODP loses, either participate in other information-based projects, or go off to focus on building (on their own) the specific bits that the large projects are lacking.
I would like to see new ways of indexing the information content of the web. But I suspect that any new way that was sufficiently user-oriented to be of interest to me (as a user, and as an advocate of other users) would be complementary, not competitive, with the existing models (Wiki, directory, search index). And if such a new way appeared, many ODP editors would be among its most enthusiastic users.
Google hasn't updated its copy of DMOZ for two years now so I doubt it still considers it "directory of authority".
Why isn't there DMOZ alternative? Simply because nobody uses directories any more - I got 11 deeplinks in DMOZ and they get me only 30 visitors a month - original long time forgotten idea behind DMOZ was surfers creating directory for other surfers not listing service. Search engines simply become too good and wikipedia took over as easiest way to find information you need.
Alternative to DMOZ is Yahoo dir which charges a fee but you can still get a listing for free if you have a good website. :-)
I think Wikipedia is a new and improved ODP really.
Dep[ends on what you viewed the ODP as - a link farm or an orgaisation of the world's information. With the latter, Wikipedia is way better.
|Google hasn't updated its copy of DMOZ for two years now so I doubt it still considers it "directory of authority". |
What about Google search? Does Google spider the current DMOZ or do they use their own (2 year old) copy instead?
I'd like to suggest that Google can be interpreted as a directory that is assembled "on the fly". It provides a hierarchical listing of companies or websites that its algo interprets and presents as if the same is responsive to the query.
Given that there are limits to the number of businesses that are online - lots of business website pages but not necessarily billions of businesses - I have never been convinced that Google should remain the be all and end all of commercial search.
In my view one of the reasons for Google's emergence was the poor execution of an online strategy of the existing entities who had expertise in commercial/business search. They simply didn't get their heads and IT management wrapped up and around the long term vitality of moving their data online. Now they are playing catch-up ball.
In my view it's still too early in the game to call a winner. Sure, there is ample proof of Google's dominance. Yet, there are some big players slowly beginning to get their act together. YellowPages.com is one example of a player that is slowly getting its game on. There's likely more to come, especially in various verticals.
So I wouldn't write off directory search any time too soon.
The very fact that some think of Google as the IT suggests that there will be any number of businesses looking at other channels. In other words, if EVERYONE is looking to be #1 in IT only so many can do that on any day. What does one do then to drive traffic?
If another search vehicle emerges that gets the job done for you then Google is what?
I'm forever amused by those prepared to call the search game over. I might be the fool in this bit of amusement, befuddled and therefore fooled by my own thinking. I'll give you that. But since I'm prepared to say I may be the fool that means I'm also willing to say that I think we're still in the first inning of the process of learning to connect with businesses online. Finding a website or a company ain't all search.
You may have noticed, for example, the push into social media marketing of business. Who needs search when "your network" can refer you to a business?
Review sites? They're still working on the model.
News media working it's way into a sustainable online business model?
CraigsList as the preemptive local city (subject matter) default search for many?
Any number of these eyeball aggregating sites may offer - or do offer, in some fashion - a version of hierarhical "directory search".
Just because they don't call it the CraigsList directory doesn't mean that's not exactly what's going on. Take another look at CraigsList and tell me what you see when you take off the "this isn't a directory" blinders.
Ditto tag clouds / Digg. Ditto a whole world of "voting directoies": "Let's get together and vote on (subject X), create and rank a list." Voila. Another directory is born and people search within it.
Why do you think Google didn't stop with its search page? Maybe Google sees the future as being a time where search is fractured, diversified, specialized.
But don't let my "1st inning" analogy - or rationalization - stop you from abandoning all hope of ever executing a plan that might work in serving the search or directory space.
[edited by: Webwork at 7:49 pm (utc) on July 9, 2007]
Just in case I've never made it clear allow me to say I do enjoy the benefits of a solid dope slapping.
I may be long winded. I may speak as if I know what I'm talking about. I may labor under the title of volunter unpaid moderator . . but that doesn't mean I am possessed of knowledge or that I may not be woefully misguided.
Just do me one small favor: Don't just call me a knucklehead when I get longwinded or "hold forth". Prove, in a compelling way, that I'm a knucklehead. It feels so much better that way. ;0)
>Why doesn't a new and improved competitor to DMOZ arise?
Maybe because 100 people working full time for a year would not be able to match the size and scope of DMOZ
>Google hasn't updated its copy of DMOZ
>for two years now so I doubt it still considers
>it "directory of authority".
Thats not stricty true. Google have not used the DMOZ RDF dump to update the categories in the Google Directory for almost 18 months (not 2 years).
Google have been using recent DMOZ RDF dumps to update the Google Directory search function - just search the Directory for a site recently added to DMOZ - it shows up in the Directory search results with a link to the category (but its not listed in the actual category).
|Maybe because 100 people working full time for a year would not be able to match the size and scope of DMOZ |
Just how long does it take to review a website? If these employees review an average of 30 websites a day, 100 people could add a million new listings per year. So I suppose you would need about 500 editors to build a new Dmoz within that time. And just two programmers.
>If these employees review an average of 30 websites a day, 100 people could add a million new listings per year.
Ah, very good, someone has Done The Math.
But don't forget this: the majority of websites aren't listable. You might have to review 3, 5, 10, or 1000 websites (depending on the category) to add one listing. The ODP gets about 10-40 suggestions for every listable site.
However, I think your programming estimates are a bit low: it's probably more like 10 man-years' effort -- so far as I can tell, that's about what it cost the ODP. That might mean Richard Stallman and a friend for a year, or 100 Microsofties for a decade. But there aren't that many Stallmans.
Dynamic search engines are easier to use than static directories.
Directories are use mostly by webmasters for links; search engines are used by regular people.
>Just how long does it take to review a website?
You are assuming thats all that needs doing. There are discussions re category structures and what goes where; taxonomy issues; there is building a consensus with the rest of the community; spam fighting; re-review of listed sites; etc etc etc
|Dynamic search engines are easier to use |
Not sure I agree with you on that point. To me the search engine SERPs are often a mess of mixed results, with unclear titles and descriptions, with spam results mixed in, with website pages ranking based upon factors that have nothing to do with what I'm looking for.
Easier to use? That's a bit sweeping.
They may be easier when I want to quickly zoom in on a particular business: Joe's Deli in Brooklyn. That is when it works. Sometimes I get 4,000 results. Sometimes the SERPs don't make it clear. Then I realize "Oh, maybe I need to refine my query" or something else.
>Dep[ends on what you viewed the ODP as - a link farm or an orgaisation of the world's information. With the latter, Wikipedia is way better.
The Wikipedia people see themselves as carefully and deliberately NOT competing with the ODP -- and, I understand, have recently modified their guidelines to make that more explicit. Like McDonald's and Citibank, "they don't sell burgers and we don't cash checks."
WP is a SUMMARY of information, the ODP is an INDEX of information SOURCES. They are quite different things, and just because I volunteer on only one of them, doesn't mean I don't highly respect the other one. (Some people are active in both.)
|However, I think your programming estimates are a bit low: it's probably more like 10 man-years' effort -- so far as I can tell, that's about what it cost the ODP. That might mean Richard Stallman and a friend for a year, or 100 Microsofties for a decade. But there aren't that many Stallmans. |
I was kinda joking, based on the number of paid technical staff at the ODP, which I thought was just 2. Correct me if I'm wrong.
Having written my own directory script (although not on the scale of the OPD), I have some insight into what it entails. Realistically you'd need a small team of competent developers, at least 2 but probably no more than 10. You can also include a lot of spam-fighting measures in the script, cutting the junk submissions and making it much easier to find decent websites to approve.
The following 6 messages were cut out to new thread by webwork. New thread at: directories/3392404.htm [webmasterworld.com]
7:05 am on July 12, 2007 (utc -5)
This thread presents the question: IF DMOZ is so bad, as any number of posts have argued, then why hasn't a competitor emerged?
What explains the absence of a real alternative?
Maybe it's easier to criticize the ODP than it is to build a better version?
Please, if you have something to add to that analysis - post it here.
If you have issues with DMOZ, per the Charter, please take them up directly with DMOZ.
[edited by: Webwork at 3:37 pm (utc) on July 12, 2007]
|What explains the absence of a real alternative? |
The amount of manpower to accomplish a large scale human reviewed/edited "general" directory is a huge task. To actually review a site, check to make sure it does not have mirrors that are already listed, and actually write a proper description showing "no" favoritism and properly done will take you 15 minutes a submission, 7-10 if you are really good and it's not a spammy category.... That adds up to a huge amount of time.
Besides, not knocking dmoz as it's the best at what it does, but general directories with a title, short description, and url are not that helpful to the masses ....
I think the next large directory will have to do more...a lot more, to attract the masses.
|Maybe it's easier to criticize the ODP than it is to build a better version? |
Yep, it sure is ;)
I have an answer an it goes like this
You might start off wanting to replace Dmoz, but the more you do it as opposed to talking about it, you quickly realise that
Either you'd rather be the next Yahoo/Google/msn
You might as well join Dmoz anyway :)
I wouldn't call Dmoz a waste of time, its a potentially valuable resource, but methinks it requires a certain mentality to want to be with them, in that particular kinda structure
We currently handle well over a 100,000 non spam emails a year.
With hundreds of emails arriving every day our email database needs a good directory structure to stop it rapidly descending into an unusable quagmire.
(Unlike DMOZ we do not have the luxury of being able to ignore/queue these requests and therefore need to keep on top of things by dealing with them in a timely manner)
And yes like DMOZ we get far higher quantities of spam to also deal with too. But unlike DMOZ this is not allowed to pollute the areas where the work is done from.
The directory structure has grown over the last 10 years and like dmoz it's many layers deep. For the last few years we have had specialist search which enables the revrieval by any word(s) in any email body or header that returns matching emails usually within 5/100ths of a second or less.
All the users also have favourites pointing to directory paths they work on a on a daily basis.
Of the three ways to access the data; directory traverse is now used less and less. Search simply for speed and accuracy has replaced the directory traverse for data retrieval.
For day to day working favourites (directory path shortcuts) enables user to target groups of data.
On large data sets other than adding, editing, removing data a directory traverse is for the vast majority of tasks an extremely inefficient use of time.
>> why hasn't a competitor emerged? <<
If it was that easy, someone from the many thousands of people who have already proposed this in the last few years would already have done it.
DMOZ adds more sites in a few weeks than <many of the second and third tier directory players> have listed in total!
Pick any category in one the other directories - compare it to the equivalent category at DMOZ -- how are they even remotely better at DMOZ when you look at whats listed (or more correctly, paid to be listed in the case of the directories you mentioned)
[edited by: Webwork at 8:44 pm (utc) on July 19, 2007]
[edit reason] Prior post removed. We're not going to discuss 2d and 3d tier directories by name. [/edit]
Folks, FYI mentioning second or third tier directories by name - and worse, adding your enthusiastic endorsement for their services - just isn't going to fly.
It's an interesting thread "as is". For all the whining and whinging I've read about the DMOZ no one has ever managed to build a better mousetrap.
That's not to say that we all don't hope for DMOZ to do an even better job than they already do. I'm sure they're working on improvements.
Anyone from DMOZ have any insights they are allowed to offer about improvements that are being worked on?
Any plans for upgrades or new procedures or improvements on any front?
|>> why hasn't a competitor emerged? << |
If it was that easy, someone from the many thousands of people who have already proposed this in the last few years would already have done it.
The only value mainstream onliners seem to see in ODP - is it's value in the eyes of G.
Once G does another spin - what value would it have?
Not Easy? it still can be achieved.
However, building a project of this scope and when done see G switching lanes, is not easy!
So why has nobody built a better mousetrap than DMOZ? Why does everyone whine and complain, but nothing better has come along?
Simple. DMOZ is the best at what it does, and I think it's the best that anyone with an all volunteer, unpaid, untrained workforce COULD do.
Most of the 'problems' that most webmasters who submit to DMOZ complain about like slow turnaround, slow or no response, unedited cats, corruption among editors, apathy among editors, etc... they all center around the editors. This isn't surprising considering that all of the editors are unpaid. It should be stated that I'm in NO WAY blaming the editors that are there and do their job faithfully and helpfully to the benefit of the directory. Soliciting volunteers is great, but they tend to not hang around for long, and the amount of volunteers you get that are really decent will be small. That leaves volunteers who's only real concern is either mucking things up, or listing only their own site. This is counter productive, of course, and only leads to tougher restrictions on editor acceptance... which leads to less editors overall... which makes it harder to get listed... which entices webmasters to apply as editors for the sole reason of listing their site... which makes it necessary to more throughly screen editors... which leads to less editors...
You get the picture.
These problems are also in no way the fault of DMOZ itself. It's my belief that almost any system that follows the same structure is doomed to... criticism, due to lack of editor support. That's what it all boils down to.
You may say that WikiPedia would have the same problem, and maybe it does, but you don't have a whole plethora of webmasters crying about it because WikiPedia doesn't exist to list websites, as DMOZ does (even if it says that's not its real job). The job of WikiPedia is to list information. I'm not trying to list my website on WikiPedia, so I'm not going to be upset if they don't list me. WikiPedia also has a great feature where, if an article has been deleted, you can view the reasons why... DMOZ offers NO feedback.
So while DMOZ may be frustrating, maddening, and even hated in some circles... I think most of us know, deep down, that any such venture to directly compete would end up almost the exact same way. As almost any DMOZ editor will tell you, it certainly isn't the tools, the site, or the structure. It's the human element that gets us every time. If you were to build a directory that is only edited by machines and tries its very best to list good sites in such a way that they're easy to find, you might have something.
Something tells me it's already been done. Maybe I'll go look it up on GOOGLE.
[edited by: Duskrider at 2:21 am (utc) on July 20, 2007]
They may center around the editors, but they really center around the expectation of a service from DMOZ. DMOZ does not provide any sort of service to webmasters. Perhaps those who want to complain should try and ask for a refund.
|Most of the 'problems' that most webmasters who submit to DMOZ complain about like slow turnaround, slow or no response, unedited cats, corruption among editors, apathy among editors, etc... they all center around the editors. |
All the complaints mention, boil down to one thing: "the editors aren't interested in what I'm interested in."
Or, which is logically equivalent, "I'm not interested in what the editors are interested in."
And ... that's OK. I think it's OK that you have interests that are different from any particular person in the world (including myself). But if you don't think it's OK, you're welcome to change. If it'd help you, I'd even forward you a list of my interests, and you can start studying up.
But it's OK if you're not interested.
The question is why doesn't a new and improved version arise, and I think the answer is because the model doesn't allow for 'new and improved'. The open volunteer editor directory structure is what it is... changes can be made, but they can only sway the overall structure in small increments before the directory becomes something else.
The model, if made strict enough to keep out junk, will always be too strict for webmasters in general... and as such will be a failure in the eyes of the majority... at least on Webmaster World.
[edited by: Webwork at 3:39 pm (utc) on July 23, 2007]
[edit reason] Keeping the focus on the issue, per Charter [/edit]
| This 71 message thread spans 3 pages: 71 (  2 3 ) > > |