|The Future of ODP|
What do you see in the crystal ball?
| 8:22 am on May 18, 2003 (gmt 0)|
It seems almost amazing that depite technical problems, often long review delays, the apparent lack of funding that ODP continues to chug along. They fixed the RDF, the site seems to be coming up faster (though still with a large number of internal server errors at the moment), and Google seems to be content using the data. Even seems like Google may prefer the unpredictability of submitting to ODP since Google doesn't like people being able to "calculate" its algo. If ya can't count on getting listed and in the Google directory you're probbaly gonna buy more AdWords at least till ya get going.
It's already probably the most valuable informational resource on the net both because of its size and the free use licence which results in most every search engine using it in some way.
Some of the RZ threads seem to result in the exclusion of some great content sites or at least deep links to pure content that would seem to be beneficial to the end user, however, from that same perspective it probably doesn't matter much if a those deep links are included or not. The good stuff will still turn up in Google.
More and more it seems like ODP is slowly going to improve its infrastructure, rewiew delays will (and have been) the norm but the whole situation isn't anything downstream search engines really have any reason to dislike. When you are selling advertising the more unpredictability there is in the free listing arena the better.
If everyone except those waiting for their sites to be reviewed/listed is generally happy with the results downstream from ODP, is there any reason for ODP to change at all or for Google to come up with any other arrangement for its directory listings?
Is ODP going to be a long term survivor?
I'm starting to think they will be, but given the mix of metas, editors and webmaster/promoters here, what does everyone think?
Hopefully we can stay clear of the "ODP must/should/will die because they didn't/won't list my site"
| 9:39 am on May 18, 2003 (gmt 0)|
It really depends on what do you mean by survive. I've personally seen people predicting the downfall of the ODP since early 1999. In terms of a project run by editors, it will always survive. Even if AOL sells it, drops it, etc, there is a core group of editors who have the means to continue on.
The larger question is will the ODP continue to be relevant. The answer to which I think will be a definate yes for the next 3-5 years, beyond which it is impossible to say.
| 10:49 am on May 18, 2003 (gmt 0)|
From the outside I don't see any reason why the ODP should lose importance. Just the opposite.
In fact at this point the Paid Inclusion Directory model seems so attractive for many medium and small portals that we will have an abundancy of those no-quality resources. The seperation between what's a linkfarm and what's a directory has become less clear all the time.
Maintaining a worldwide all purpose directory is extremely resource intensive and difficult. Currently I don't see any big player who would have the means and the overwhelming demand to pull that off.
The only thing that could bring the ODP down is the ODP.
| 11:20 am on May 18, 2003 (gmt 0)|
I'm pretty much in accordance with the opinions above. There is such a momentum, albeit an unrushed and methodical one, that the ODP directory is absolutely here to stay, as a valuable and unmatched resourse.
AOL are not, IMHO, the right parents - they've been starving it and keeping the kids happy with 'wait and sees' and 'soon dear, soons' for too long, but there does seem to be a turn in the right direction over the last few days, so we'll see, maybe it'll all come good.
On some fronts I feel that a degree of automation has to be bought in - not on adding listings, but blocking them. The unreviewed queues are often unloadable for anyone on dial-up, and it takes a dedicated soul to wade in and slash out the swill of cookie-cutter, re-direct, mirror, affiliate/booking lookalikes.
(Cue the 'but that's exactly the problem with DMOZ' flames. As with Google, it's all about content and the end-user, simple.)
Maybe a policy change will occur to obviate this, but, if not, something will need to be done to stop flooding, I feel.
| 11:20 am on May 18, 2003 (gmt 0)|
So long as Google uses the ODP as its directory, the ODP will be important.
| 11:24 am on May 18, 2003 (gmt 0)|
Oh I think the ODP goes a fair bit further than Google. If G just continues to crawl it, it'll be important ;-)
OMG yes! - They have not done well.
BTW, excellent post skibum!
| 3:18 pm on May 18, 2003 (gmt 0)|
I agree with rfgdxm1 and Nick_W.
I don't use ODP myself. I didn't think anyone did except SEOs and wannabe directories using its dump.
| 7:14 pm on May 18, 2003 (gmt 0)|
>It's already probably the most valuable informational resource on the net
I agree with that 100%.
Having said that I think in the last 12 months things have taken a turn for the worse. I personally detect an attitude that is hostile to the SEO/Webmaster which imho does not bode well for the long term future of the directory. It almost seems to me that "they" are trying to fight a battle with submitters, that mind set is wrong because it is a battle that is unwinnable. I think they have locked themselves in to a "we are the good few" fighting a battle against the hoards of evil[:)], wrong way to play the game.
I think very fundamental mistakes have been made of late, I'll give you a few examples.
The dumping of submissions whose IP could not be determined into a "holding pen". Many [most?] ISP's operate through proxy's, it looks a very lame way of trying to thwart mass submitters to me.
The RZ thing troubles me greatly, it could be read as an avoidance of responsibility by the directory and as far as I can see serves no useful purpose.
The latest purge of sites as seen here If in doubt, throw it out" is IMO a pretty fair statement of what we intend for sites offering online hotel reservations [webmasterworld.com] chills me to the bone. A good directory would look at the same problem with a different viewpoint. They should be saying "we are not serving our users well, we need to up the quality in those cats", that involves finding the quality sites in that area not just deleteing the bad ones.
I think DMOZ is here to stay, lately I think that is because of the paucity of the competiton more than anything else.
| 7:22 pm on May 18, 2003 (gmt 0)|
I think the ODP will continue to get squeezed from both sides. On one side, search engines keep improving doing away with the need for basuc categorized lists of sites. On the other side, there are more a more speciality directories that have fields particular to their subject matter.
As rdf and others stated, the ODP will still have utility as long as it is supported by Google, but its importance as a resource in its own right will continue to decrease over time.
| 7:39 pm on May 18, 2003 (gmt 0)|
|The RZ thing troubles me greatly, it could be read as an avoidance of responsibility by the directory and as far as I can see serves no useful purpose. |
A responsibility to whom? The directory isn't really responsibile to anyone but the users of the RDF dump, and end users. As for a useful purpose - I would think the amount of traffic indicates some people think its useful.
|The latest purge of sites as seen here If in doubt, throw it out" is IMO a pretty fair statement of what we intend for sites offering online hotel reservations chills me to the bone. A good directory would look at the same problem with a different viewpoint. They should be saying "we are not serving our users well, we need to up the quality in those cats", that involves finding the quality sites in that area not just deleteing the bad ones. |
In most cases I would agree with you 100%, but in the case of hotel reservation directories there are no quality sites - or at least that are findable anyway.
| 8:34 pm on May 18, 2003 (gmt 0)|
I think the ODP faces several challenges, including:
1) The sheer size of the Web, which grows faster than the ODP can recruit and train competent (and committed) volunteer editors.
2) The increasingly commercial nature of the Web. (How many people think it's really, really fun to edit business directories?)
3) The improving quality of spidered search engines. (For most topics, it's easier and quicker to find quality sites in Google than in a massive multilevel directory like the ODP or Yahoo--and Google's listings will be more up to date.)
4) Adequate financial support.
5) Relevance. If the ODP were to disappear tomorrow, would anyone really notice except for Webmasters and the search engines that use it as a free source of directory data? Is there the same kind of need for the ODP as there was back in 1998, when--as "NewHoo"--it had the goal of beating Yahoo (which was then perceived as being inadequate) at the directory game?
| 8:39 pm on May 18, 2003 (gmt 0)|
>A responsibility to whom? The directory isn't really responsibile to anyone but the users of the RDF dump, and end users
I know that, you know that, just pretend the webmasters/SEO's are important, life will be easier I promise.
>but in the case of hotel reservation directories there are no quality sites - or at least that are findable anyway
I like a challenge, if you would like to sticky me the cat in question I'd love to give an independent view.
| 1:35 am on May 19, 2003 (gmt 0)|
ODP will thrive going forward because the long-term underpinnings of Internet sociology favor an organization with high integrity.
ODP runs according to the principle of rationality. Itís a stark contrast to Internet organizations that appear to be like high schools. Places like Yahoo and Looksmart and MSN and Google Ė deal with these people and itís like some adolescent nation: the drama nerds and the band people, the cheerleaders who are only interested in dating the football player with the biggest muscles. The student government wonks who just ooze sliminess. These are organizations that run on emotion and human frailties. ODP isnít like that at all. Everything makes sense at DMOZ.
I, and many others, have noticed very vocal discomfort with ODPís management and whining about alleged abuses. But this is just sour grapes. You frequently see people who own substandard websites complaining about ODP, as if the failure of their site to be listed was in someone elseís hands, not their own. These people are too weak to admit their own failings. They should take responsibility for their own lives. Only you can help yourself.
ODP is a perfect meritocracy. Unlike corporations or sports teams or school fraternities, advancement at ODP comes strictly from hard work and quality contributions to the group. Nobody gets ahead because they have bigger biceps or because their Dad knew somebody twenty years ago or because they kissed somebodyís behind.
What bothers people about our organization is that they arenít comfortable with a group with such high integrity and commitment to quality. Nobody can come in and buy their way into the directory. Nobody can use flattery or favors from some old-boy network to influence the directory or the search results. Weak-minded people canít fathom or cope with such a straightforward system, so they attack us. Attacks on all sides by people who canít handle the truth. We will always have the philistines to grapple with.
Yes, it bothers us that AOL thinks they are in charge. AOL is probably just like any other corporation, with favoritism and politics. Everyone who cares about the Internet should be glad that AOL has a hands-off policy when it comes to the directory. Itís best to think of AOL as a necessary evil.
My predictions: Internet users (the ones who matter anyway) will come to understand Google, Overture, Yahoo, Micro$oft, et al are in it for the money. They are low integrity organizations answering to aging frat boys at Wall Street investment banks. Only ODP has the integrity to build and maintain the webís best directory, to be the organization to choose the best sites based on quality and contributions to the Internet as a whole. When people come to see that ODP editors are the cream of the crop, a natural aristocracy arising from the masses of Internet users, they will appreciate the outstanding work these hard-working, sincere, virtuous heroes have done in producing a resource for humanity. Eventually most of all major search portals will use DMOZ, and the participation in the project will rightly be regarded as a source of pride. I also wouldnít be surprised to see ODP win a Nobel Peace Prize some day, much as Doctors Without Borders did a few years ago.
| 2:32 am on May 19, 2003 (gmt 0)|
That's an interesting way to look at ODP, Aaron, never quite seen it described in terms quite like that before.
The AOL hands off policy defineitely makes it a dramatically more credible source of information as does its willingness to not only list the McDonalds and Monsantos of the world but the McSpotlights as well.
|Eventually most of all major search portals will use DMOZ, and the participation in the project will rightly be regarded as a source of pride. |
Eventually? Seems like it would have to be regarded as a source of pride now in order to stick with it for a while, especially for anyone who hits the meta level or has a bunch of categories to take care of.
Interesting viewpoints all around.
| 4:25 am on May 19, 2003 (gmt 0)|
The ODP is currently the best thing out there; that's not saying much, though, when you consider that it's competition is Yahoo!, Zeal/L$, GoGuides and JoeAnt. None other those directories can compare to the ODP at this time. But how hard would it be to beat the ODP? With deep pockets, and the backing of several online communities, and a rock solid branding plan, a new directory could easily take advantage of the anti-ODP sentiment that fills many webmaster forums (due mostly to anti-webmaster setiment from meta editors); and once something better is on the horizon, ODP would be last week's trash. Especially if the new directory started out with PR8, and had a thoroughly open and democratic review process for rejected sites.
Just my opinion, of course.
| 5:08 am on May 19, 2003 (gmt 0)|
Good thread Skibum
It was refreshing to see a directory thread NOT turning into an ODP bash!
As long as the web community help to promote the ODP and directories in general as valuable resources -- but those resources must aim for quality over quantity -- I see a long life ahead for the ODP.
| 7:36 am on May 19, 2003 (gmt 0)|
Interesting thread Skibum, and agreeing with richardb it's refreshing to see it NOT turning (at least, yet) into an ODP bashing one.
I'd like to add my 2 cents, being in a position to look at the ODP present and future from several points of view.
Rafalk is right when he says that the larger question is will the ODP continue to be relevant. But here (as well as in any discussion about relevancy) we can easuly see that this term can be used to express both:
a) relevancy for the final user - being a comprehensive and quality collection of sites that downstream SEs and portals could continue using (with or without integration with proprietary algos & ranking/sorting systems) to serve relevant results to searchers.
b) relevancy for webmasters/SEOS - being an important (whereas not the only, and probably often overestimated) step to consider in an effective website SEO strategy.
Answer to (a) is a definite yes. The ODP will continue to grow and improve, since (and until) its primary goal is collecting and describing quality resources for the final user. I actually don't see any other player around that could find and develop the resources needed to challenge the ODP preminence.
This doesn't means, of course, that the ODP is -- or will ever be -- perfect. There are still, and there will always be, issues and problems to be solved, but the ever-evolving nature of this project has proved to be able to face them and deal with them in an effective way, notwhitstanding predictions of downfall since its very beginning and up to the present days. Notice that there is an evolution of the "ODP model", both on the management side (once born, several Staff employees were in charge of most of the tasks which are now performed by a steadily growing number of meta editors, thus shifting the whole model more and more towards a community-managed system) and on the technical side (issues like spam, abuse, unrevieweds backlog are taken very seriously, and whereas the actual response to constructive criticism -- both internal and external -- isn't always as fast as people would like, the ODP is really not that far behind what's needed to face the evolution of the Web.
As already said, the AOL hands off policy does more good on the side of credibility than bad on the side of technical support. And, even if AOL will ever drop the ODP, there is a core group of editors who have the means to continue on.
Answer to (b) is more unpredictable. Looking at recent threads, it's clear that several webmasters/SEOS consider the ODP the most important (when not the only) step to be considered in an effective website promotion strategy. This is just plainly wrong, as explained multiple times, but looks like that the old "if there's queue outside, if people talk about it, the place MUST be trendy, thus I cannot live without being accepted in" is difficult to eradicate from most people's minds.
Realistically speaking, ODP is one of the many steps to be considered by a webmaster/SEO having a quality and content-rich website to gain more relevancy in SEs. It's a link from a quality resource, not more and not less than that. Splitting hairs abouut how the fraction of PR passed from category X couples with the relevancy of keyword Y placed in the site title is an interesting SEO exercise, but doesn't change much the facts.
Sure, it's still quite easy to see no-content doorways ranking top in several SEs for competitive kwds, whereas these same pages/sites don't "benefit" (and shouldn't/won't benefit) from a listing in the ODP. It's up to SEs improving their algos to serve more relevant results to users (some of them are actually working this way), and when SEOs will be forced to face up that optimizing means serving to SEs something that will put them in a position to serve their users with the exact stuff users are looking for, this will probably indirectly add up to the relevancy of human-edited directories/collections/resources in general, and (again) to the ODP in particular. Even though, and while I agree with Aaron Larson that a site's future is in the webmaster's hands and not in someone else's, I still don't forecast neither a boost nor a loss of relevancy for the ODP, which is born, has evolved, and should continue to be considered just for what it is: a comprehensive collection of quality sites, where quality sites are accepted and others don't.
Interesting scenario -- notice that I neither wish it happpening nor I think it's realistic, just pointing out that it could have even positive effects -- would be if Google (and why not, all other SEs currently using ODP data) drops the ODP. Will the ODP become less relevant. From a SEO perspective, probably yes, but just a bit. It'll still be a good link from a quality resource. From the end user's perspective, probably not at all. Sure there will be less final users indirectly benefitting from ODP-served data (the very few users who directly use the ODP won't probably notice it, but they are, after all, very few).
OTOH, this scenarion opens interesting possibilities for speculation. Will the ODP die if Google drops it? I don't think so. Several editors, myself included (that is, most of the "active" ones), will continue to edit there, reviewing and categorizing sites. A possible effect will be that most webmasters/SEOs who currently edit there will probably reduce their activity, or even quit. Most webmasters/SEOs who are now striving to have a site listed, and complaining when it isn't, will stop holding their breath and move forward to other more "interesting" stepping-stones. Which gives us less spam submitted, lower unrevieweds backlog, less resources needed to watch for possible abuse, more or less the same number of active editors working (the "who cares about other SEs do with our data" attitude is actually more sported by active editors than by most less active ones, those who -- oh surprise -- see the ODP as a stepping-stone for their sites), less complaints and time spent answering complaint, smore or less the same "growth ratio" (already said multiple times, whereas submissions are a good way to feed the directory, editors can and do find new sites following links from currently listed ones, or searching in SEs -- they're actually encouraged to do so in categories where there are no unrevieweds). Think of it as sort of, hum, a "sabbatic period" where a more thorough cleanup could take place, but which won't dump the relevancy of the ODP as a comprehensive collection of quality sites, and won't change its staying ahead of any other player. That is, if the ODP will ever be dropped by major SEs, it will only become after some months more relevant, clean, comprehensive, and "attractive" for the same SEs who dropped it.
For these reasons, yes, I do think that ODP is going to be a long term survivor, and while not perfect as for what concerns room for improvement, it will continue to be the most relevant (absolute value) resource available.
| 1:05 pm on May 19, 2003 (gmt 0)|
Good post, ettore.
I predict (and this is just me talking) that at some point submissions will be turned off to ALL spammy/SEO-hammered areas, and editors will be allowed to actually find useful, high-quality sites all by themselves. At the moment there are category trees with jillions of barely passable sites waiting for review while their owners chafe at the long wait for their sites to get reviewed (when they mean LISTED) and simultaneously post in forums such as these that the ODP is dead. This whole circus really has to end if editors are to keep their sanity and the directory is to grow in a healthy way.
| 2:16 pm on May 19, 2003 (gmt 0)|
|...at some point submissions will be turned off to ALL spammy/SEO-hammered areas... |
I'm a bit baffled as to why "spammy" and "SEO-hammered" go together in so many editors' minds, though this might just be a semantics thing.
This tactic would make sense only if there is no hope of finding ANY content-filled sites which belong in the cattegory.
|...editors will be allowed to actually find useful, high-quality sites all by themselves. |
That's the way it used to be, but we only did that until we were well-known enough to get people submitting.
When people actually began submitting, it was seen as an improvement. Times change, I suppose. :)
Incidentally, many editors DO still find useful, high-quality sites all by themselves. There are many editors who find more sites by themselves than they do reviews of submissions.
As to the original question, I have always been a firm believer that the more search engines there are, the better the web is for those of us searching for information.
After the magnificent losses AOL posted this year, I would be surprised if they added any staff to the project, but it is in their best interest to keep it around as a resource for their own search engine.
Google, too, would do well to keep the directory, and while I believe it's an excellent jumping off point for Google's spiders, I believe the weight ODP has in relation to PR will be minimized in the future, due in part to the increasingly exclusionary policies in all categories.
| 2:41 pm on May 19, 2003 (gmt 0)|
>>ODP is a perfect meritocracy. Unlike corporations or sports teams or school fraternities, advancement at ODP comes strictly from hard work and quality contributions to the group. Nobody gets ahead because they have bigger biceps or because their Dad knew somebody twenty years ago or because they kissed somebodyís behind.
That is one of ODPs biggest problems. Advancement comes from quantity of contributions, but somewhere along the line has got lost the original thought that experts could be attracted to edit local topics or specialist topics.
Too much politics going on there today to make this a reality
>> The sheer size of the Web, which grows faster than the ODP can recruit and train competent (and committed) volunteer editors.
Second problem - ODP can never attract enough editors. Unreviewed still there like an Albatross
>>They should be saying "we are not serving our users well, we need to up the quality in those cats", that involves finding the quality sites in that area not just deleteing the bad ones.
It seems to be an admitance of defeat if the "new" policy of dmoz (if in doubt, throw it out) is here to stay.
>>I think DMOZ is here to stay, lately I think that is because of the paucity of the competiton more than anything else.
Cost, time and paucity of decent volunteer editors make this true today. I have no connection with JoeAnt, but they continue to build a decent directory
However in the long term I continue to believe that mini specialist directories will be the way forward - hotels, real estate agents, cars, and the other products advertised today in the press. As this seems to be the only way that the punter can get decent (spam free and comprehensive) information on the subject of their interest
| 2:45 pm on May 19, 2003 (gmt 0)|
>The dumping of submissions whose IP could not be determined into a "holding pen". Many [most?] ISP's operate through proxy's, it looks a very lame way of trying to thwart mass submitters to me.
Here's a different perspective, from someone who spends most of their online time with at least one browser window dedicated to pushing stuff through the "bin" (actually it's more like a "chute":
1) The initial software was based on using IP addresses (rather than something like cookies) as the "state" for the form processing. Submittals from indeterminable IP addresses were placed in the bit bucket...yes, discarded completely. The submitter was given a message "to get this site submitted manually, send e-mail to ..." The current state is a successful (but partial) automatization of that old process, and a step in the right direction; we are obviously retaining many more of those submittals than the original system did.
2) It is true that the general internet trend has been towards increasing use of proxies (and not just by ISP's -- corporate sites need them even more desperately), and so the IP-determining problem has gotten more acute in the meantime.
3) The current system is not ideal, not intended to be the final step. Part of the reason for the manual intervention is to help get a feel for what really needs doing, (including what spam problems there are); and there is discussion about how to improve it, perhaps to the point where no manual intervention is needed.
4) Obviously, we'd all like more technical support. But I'm sure, given whatever level of support there is, you guys want them getting the new servers up as a first priority. (And I wouldn't mind that myself).
| 4:35 pm on May 19, 2003 (gmt 0)|
|That is one of ODPs biggest problems. Advancement comes from quantity of contributions, but somewhere along the line has got lost the original thought that experts could be attracted to edit local topics or specialist topics. |
Advancement has never been about quantity but rather the quality of an editor's work. The specialist knowledge is very useful in matters of taxonomy, but when you get down to it the most important point of an editor's job is to accurate discribe sites. If an editor can't write a decent title and description then there's no point in giving them more permissions. Politics have nothing to do with it.
|It seems to be an admitance of defeat if the "new" policy of dmoz (if in doubt, throw it out) is here to stay. |
Please. It's not a broad policy like you're implying it to be, but rather a practice used for an incredibly narrow subset of sites.
|However in the long term I continue to believe that mini specialist directories will be the way forward - hotels, real estate agents, cars, and the other products advertised today in the press. As this seems to be the only way that the punter can get decent (spam free and comprehensive) information on the subject of their interest. |
If done well, such directories could be very valuable resources. Unfortunately nobody has come up with a good way of doing this yet. In most cases, these directories are in the business of making money - which leads them to include an incredible amount of trash.
| 5:32 pm on May 19, 2003 (gmt 0)|
>>Advancement has never been about quantity but rather the quality of an editor's work.
Yes, I know, I will admit to being provocative in my choice of wording.
However it is a serious point. Correct me if I am wrong, but I thought that the original intention was that "experts could be attracted to edit local topics or specialist topics".
>>If an editor can't write a decent title and description then there's no point in giving them more permissions.
That is patently true...however ODP has somehow lost (again corrrect me if I am wrong) the way to attract experts to niche categories - rightly or wrongly, a brain surgeon feels miffed if their editing of that specialist niche is re-written by a more senior editor.
I would accept that ODP probably could never attract enough experts for niche categories. Therefore you have to attract "general" editors wioth time to spare for quantity (and quality) editing. Problem is that there are just not enough altruistic guys out there.
>>If done well, such directories could be very valuable resources. Unfortunately nobody has come up with a good way of doing this yet.
I don't know about that, tos do not let me go any further on WebmasterWorld ;)
| 6:10 pm on May 19, 2003 (gmt 0)|
A brain surgeon might know of some great websites -- that's where the expertise lies. They may be awful typists and/or description writers. Hopefully they're knowledgeable _and_ good writers, but those qualities aren't sold as a set ;)
<added> Laisha, I was just speculating after gazing into my own crystal ball. The odds are that it won't happen, but I do think there's a little bitty chance that it will.
As for SEO-hammered, I mean those folks who are in the business of getting (high?) placement of their clients and oversubmit to multiple categories - some correct, some not - at multiple times until something "positive" happens. Not everyone is like that, but I'll give an example which has been perplexing lately: someone(s) in China believes that they can submit Chinese textile factory sites to just about every category with the word "business" in it. They submit a lot of sites and fairly often. Perhaps they're abusing some auto-submit software, but we can't be sure. It's not true junk as the sites are what they say they are, but they are all in the wrong places to get anything listed quickly. This stuff clogs up the workings for everyone.