Forum Moderators: open
Its ran by volunteers mostly and it's backed by AOL... They just fund it.
My question is why dont they use it for there search results?
"and the ODP is an excellent directory.."
Well, some parts are. The Science, Arts, Reference, and a few others are pretty good. However, the Business, Computer, Shopping, and many other categories are many months, even years behind. There are over a million unreviewed in ODP, and probably half of those are in the 3 categories I mentioned - so those are not so good.
The problem is that the interesting categories, such as Science, keeps editors interested. It is difficult to stay interested in tracking down the 97th spam submission from affiliate xxx-drugstore.com
...that's what my logs are showing for a niche e-commerce site.
It is difficult to stay interested in tracking down the 97th spam submission from affiliate xxx-drugstore.com
Yeah, that is a real problem. But take a look at this:
[dmoz.org...]
For some things ODP does an incredible job.
Believe or not, I actually got a hit from Webcrawler the other day. I can recall a day when them and Alta Vista were top dogs.
DMOZ is great in a few areas, horrible in others. It depends tremendously on the editors in that section. Before I got in a tiff with them and we parted ways, I had my cats in /Science good shape, but other cats just a link up were a total mess.
But that is part of the problem - interesting categories can get editors to stick around, others cannot, so they suffer.
And the spam rate is only going to get worse - there are hundreds of thousands of domains going dead each month now, and tons of them are being bought up by porno and link spamming places. I even see it affecting Google to some extent - once in a while I will click on a link, and go to that damned page that is nothing but ad links. But Google updates a lot faster and cleans out the garbage fast.
I think it is just a matter of time before DMOZ is forced to go through some kind of pre-screening submission or they are going only get more behind. That I think is the real reason that Yahoo went pay - just to cut down the spam submissions on drug-affiliate-237.com. DMOZ cannot go pay, but they could probably make the submission form more stringent.
>>I got in a tiff with them and we parted ways<<
This is not a true statement.
</off topic>
ODP will never generate any revenue. See [dmoz.org...]
A subsidised affair, out of date that will become a liability (?) :)
When I assist people in promoting their site on the net, I get blank dear-in-headlight stares when I SAY "dmoz.org" .........I then have to write it out: d m o z . o r g for them to comprehend (and still that may not work)
Exactly.
And something that costs you money and generates no revenue is, in technical terms, called a "Liability".
A liability is something that a company can put up with in some cases, if it also generates an offsetting amount of good will, publicity, or something else. The question is, is AOL getting it's moneys worth?
If AOL decides it is not, it could well just cut DMOZ loose, or sell it - but I am not sure who would buy it if AOL does not want it.
The fact that AOL started with around 6 paid staff, went to around 25, then very shortly after that cut it down to two might give some indication of how much AOL thinks it is worth vs what it is costing them.
More and more, I question the viability of any of the current directories - which is mainly DMOZ and Yahoo.
It would not surprise me a bit to see Yahoo start charging for ALL submissions - perhaps a nominal $10 for charities, and a graduated scale up from there for others. Not because they need the money - but because that would cut down the number of submissions tremendously - and spam submissions even more.
DMOZ does not have that option. In fact, DMOZ as it is currently set up has pretty much no options at all...
"Dmoz is to the net what the BBC is to the UK.."
-- hehe, good analogy.
[1] That's what I paid in February. It might have gone up since then. Also, some might rub off on the collection agency en route.
[2] They have detector vans armed with the ability to detect operational TVs and lists of non-license paying households. This was even before 1984.
If AOL decides it is not, it could well just cut DMOZ loose, or sell it - but I am not sure who would buy it if AOL does not want it.
Basically what I was meaning was "X entity" pays for something like DMOZ / BBC.
I don't knock the BBC too much, they do a job, so does DMOZ. The public use them because they are free. But they are a burden to those who sponsor it. Same goes for the UK NHS ;)
The only good free service is a paying service :) or something to that extent. Well, when it comes to the likes of DMOZ and its lack of freshness - "revenue" is certainly a keyword.
Do THEY have a "donate" button? :)
First, the number in the queue is not that high, second, while the number of direct 'hits' from ODP is small (always was, always will be - planned that way), the number of hits attributable to ODP (via Google and others) is high and growing.
Second, the BBC never, ever had detecting equipment in those vans - all they had was
1. A rotating mock arial
2. A copy of the electoral roll (list of voters/residents)
3. A list of license owners
4. A human being comparing the two.
Now the van contains the same items, with a more realistic-looking rotating ariel, and a computerised set of lists.
And they've always had very good publicists.
I would say that 1,000,000+ is sorta high. That his nearly 1/3 the directory size. And if you take out all the errors, dead links, spam links, mirror sites, deep linked sites etc. it is probably closer to 50%.
Granted, probably 50% of the unreviewed are multiple submissions, spam etc - but it actually takes longer to go through those than it does legitimate sites.
In November of 1999, the total unreviewed was under 5,000. If the growth rate stays constant (comparing Jan 2000 until May of 2002), by summer of 2003 there will be more untreviewed than there are listed sites. It may have slowed down some, I really don't know. I was told a couple of months ago that the number of unreviewed now goes up about 2000 per day.
repeating a hundred times an incorrect statement, and through different boards, doesn't make this statement correct. That is, understanding the unreviewed count is far less easy than what could appear to inexperienced editors, or outsiders making guesses and projections. The number of currently unreviewed sites isn't 1M+, but is close to 900k. The 1M+ count would be correct if you consider Test and Bookmarks categories, which are there for completely different purposes than making sites available to the public and where unrevieweds pending are mostly sites already listed in the main ODP.
>> probably 50% of the unreviewed are multiple submissions, spam etc
In my personal experience, inappropriate submissions, spam, dups, etc. account for nearly 70% of the current unrevieweds. Consider that this rough percentage may vary depending on categories/areas: some spam-magnet ones get less than 10% guidelines-compliant submissions.
In addition, some World/ categories where there are only a few editors experience a high number of unrevs, thus raising up the overall percentages and numbers. The current *total* unreviewed percentage for the English part of ODP is less than 31%, thus leaving us with only 10 to 15% of guidelines-compliant, "good" sites waiting to be listed.
Also consider that a certain percentage of the current guidelines-compliant sites sitting in unrevieweds are pending double-listings (i.e. an editor in /Widgets or World/[Language1]/Widgests lists site XY in the category and sends a copy to World/[Language2]/Widgets since the site has a version in [Language2]). This means that site XY is already listed in its primary language category, and just waiting do be double-listed (or triple, quadruple, etc.) with its different language versions. From a visibility point of view, this makes XY already available to the surfers (and spiders using dmoz db).
In the meanwhile, most World categories where the unreviewed problem is far more high, are experiencing a rapid growth in *listed* sites (which is far more high than the growth percentage of listed sites in English categories): yes, ODP is slowly becoming more and more "international". For the curious, W/Espaņol reached a 4% monthly growth rate, alongwith W/Chinese and W/Dansk; 6% for W/Italiano; 12% for W/Catalā...
This said, the "unrevieweds problem" is taken very seriously. A few tools have already been implemented, alongwith special permissions that put editors in a position to deal with unrevieweds in categories where they don't have editing privileges. Sure we need more tools, and a few suggested ones are under study or test.
In any case, the solution to the problem lies in a better *filtering* system, where the balance between automated filtering and human review is properly set to a point where the "big spam" can be weeded out, and editorial activity can be focused on what's left. It's not an easy task, and it will take some more time.
Frankly, I think the forecast of more unrevieweds than listed sites by summer 2003 is neither accurate nor something that will ever happen.
So, you are saying that the 300,000 entries that people have in their bookmarks and test categories do NOT show up in the totals for DMOZ's total number of sites, but the unreviewed in those same categories DO show up in the totals? That seems rather inconsistent.
"unreviewed sites isn't 1M+, but is close to 900k...."
I would say that in terms of how long it will take to clear that backlog, the difference between 900k+ and 1000K+ is not all that relevant. And that munber is going up by 1000 to 5000 per day, depending on who you listen to.
In the last major category I worked on which had lain totally untouched since Feb of 2001, about 30% were dupes and spam out of some 1950 sites. About 25% were in the wrong category, which just adds to the workload - when I moved sites to the correct one, someone still had to take care of it.
"The current *total* unreviewed percentage for the English part of ODP is less than 31%,.."
As of two weeks ago, Business had 80%, Computers 71%, and Shopping 59%. The Science, which is where most of my categories were, was around 18%. Those first 3 categories accounted for about 1/2 of the total unreviewed.
I have had zero experience in the world cats, so will take your word on that.
As I noted, the projection for summer of 2003 was if the rate remained the same as it has in the past 16 months or so. I also noted that this has apparently slowed - probably due to several factors, included a large drop in the number of new dot.coms and the huge number of expiring domains. Even if the current rate of 2000 a day is correct, as I was told a while back, is true - that is still around 800,000 more unreviewed by next summer.
Add to that the fact that according to SE World, more than 25 million domains will expire this year (between May and Dec) - which means a lot of weeding out as sites go dead and add to the workload.
I am aware that some new tools were being worked on, but my personal experience with one, Greenbusters, was not all that favorable, and I also saw very little interest in it from midlevel editors.
Perhaps they will have to enact stricter standards just to slow down the submissions eventually, since they can't charge for them. Personally, I would have been happy to not allow any site with more than one pop-up to be listed.
Remember that some editors, especially those who are fairly new, will deal with the 'nonspam' with confidence, leaving the spam for someone else; such categories may have plenty of unreviewed sites ... but 95%+ will be spam, deeplinks and duplicates.
Also remember that some of these tools can be used among accepted sites; so many editors are spring cleaning sites that previously slipped through the net - and now shine out like lighthouses.
This is not exactly 'taking an eye off the ball' but it also explains why some short queues have got a little longer; just for a while.
In short, feel free to whine about ODP, and predict it's overloading / demise - you ain't the first, you won't be the last.
Meanwhile, in the real world, momentous anti-spam measures are happening almost as we speak, and editor time is increasingly productive time.
If you know of a better human directory, I'd be impressed. ;)
I am not predicting the total collapse of ODP, Yahoo, or any others. I am predicting that they will have to make some drastic changes in order to stay relevant. Surviving and staying relevant are two different things.
WebCrawler has survived, but few would call it relevant or important. It failed to change with the times.
For example, ODP may eventually have to split into 2 or more totally different directories.
