homepage Welcome to WebmasterWorld Guest from 50.19.144.243
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Search Engines / Directories
Forum Library, Charter, Moderators: Webwork & skibum

Directories Forum

This 103 message thread spans 4 pages: 103 ( [1] 2 3 4 > >     
DMOZ is getting smaller
Dead links being deleted faster than editors can add new sites.
lgn1




msg:3381256
 3:46 pm on Jun 28, 2007 (gmt 0)

Im not sure if this applies to DMOZ overall, however it
does apply to the Shopping Main Group.

Shopping appears be be shrinking at 1 to 2% per year.

In my little neck of the woods (Dmoz subcategory), I used to have 76 competitors a few years ago. It has now been reduced to 46.

Everywhere I look, most shopping categories are looking for volunteers.

I know that bigger is not better, and etailing is a lot more difficult now, than it was in 1999, but is this an indication that etailing is being consolidated amonst fewer players.

Or this just an indication that DMOZ has a lack of editors and they are unable to find quality sites faster, than sites are going out of business or being bought out.

 

cbpayne




msg:3382801
 10:27 am on Jun 30, 2007 (gmt 0)

>Dead links being deleted faster than editors can add new sites.

Thanks for pointing out this really good piece of news. Shows the quality control is working! I am intrigued as to why you think its somehow a problem.

Despite clearing out all the deadlinks DMOZ has had a net increase in sites listed of 2467 sites during the month of June. How many other directories can claim that incredible level of growth?

DMOZ is NOT getting smaller.

lgn1




msg:3382886
 2:02 pm on Jun 30, 2007 (gmt 0)

Does DMOZ editors have access to these trends? DMOZ appears to be growing overall however the DMOZ shopping category is shrinking.

I have heard that it is very difficult for a new editor to get a shopping subcategory, and their are few editors in this category.

I realize that shopping is probably one of the categories most subject to abuse, and may have a higher standard.

Anybody have historic monthly figures on the DMOZ shopping category, or does DMOZ only keep track of the grand total across the site?

windharp




msg:3382905
 3:00 pm on Jun 30, 2007 (gmt 0)

Does DMOZ editors have access to these trends? DMOZ appears to be growing overall however the DMOZ shopping category is shrinking.

Statistics covering directory wide numbers used to be published on a monthly basis before the crash. See [freenet-homepage.de...] for details.

Unfortunately, AOL Ops is still working to restore our toolbuilders access to the needed resources, so a lot of things still can't be done properly.

The ODP pages do have counters, so whoever wants to can monitor growth. We used to have tools to do monitoring of toplevels, but unfortunately most of them aren't available at the moment. Reason see previous chapter. But: Yes, in theory we do have access to those numbers.

When comparing these numbers, please keep in mind that due to tool runs (details see remarks in the abovementioned ODP Reports) some weeks show reductions in listed sites, while others show growing numbers. Until graphical views are available again, only longer periods of time (like one months, half a year, ...) give valid estimates.

Anybody have historic monthly figures on the DMOZ shopping category, or does DMOZ only keep track of the grand total across the site?

We usually don't publish statistics for parts of the ODP. Experience shows, that this type of information leads to "I submit my site to the category with the most additions intead of the best suited one, because this way it is listed faster" behaviour. (Which is b***s***, because it simply increases editor workload by adding "1) search correct category, 2) move site for review in the correct category" to the workflow. And if the editor is in a bad mood, he won't bother with moving an adult toy shop from the horsebreeding category.).

But of course, anbody with a broadband internet connection and rudimentary progeramming skills could download the RDF files for two dates and do a comparison. Or - if you are not interested in to many details - you can use the wayback archive.

Whatever you do, please compare numbers you got the same way. The on-site counters tend to be different from those generated by counting entried in the RDF file for soem reason.

Looking at a few numbers (for example 120k back in 2003 and about 109k now - data was taken from the same source), you will notice that in the overall trend, shopping does loose size. Looking in more detail, one can notice that RoboZilla (our automated linkchecker) sorts out about 3k sites in every go (usually several times a year). Other tools remove sites as well.

From my experience there are a lot of reasons making a shrinking of Shopping more understandable. Let's face a few of them.

1) Editing listings for shopping sites becomes incredibly boring over time. Ever looked at 100 sites selling absolutely the same stuff, tried to find the differences, and come up with a good description for each?

2) Experience shows that most people pretending to be interested in editing a shopping category are not really. They are just interested in "Make $$$ fast". Of course there are exceptions, but they are the minority. (Brings me to what you have hears: Yes, it is more difficult to become an editor in a Shopping category. I don't think anybody needs a more detailed explanation why, huh?)

3) Our listing criteria changed over time. Nowadays we have much more strict guideliens than a few years ago on affiliate sites, product subsites, and other sites you would typically find in the Shopping branch. In a human edited directory like DMOZ, it takes many years till the remains from the olden days are sorted out.

4) Sites shut down a lot faster in Shopping. THey simply stop to exist, are bought by other companies, stop selling stuff, or disqualify as a shopping listing in another way. I don't have numbers to prove that, it's just my personal experience from a few years of editing. Thinking about it, the difference to the normal branches most likely are tge sites of those "Make $$$ fast" people who at some point realize that it's pretty hard to do that in the internet. :-)

I could go on with many other reasons, but from my point of view those are the most important.

cbpayne




msg:3382907
 3:08 pm on Jun 30, 2007 (gmt 0)

Does DMOZ editors have access to these trends?
For another reason I went throught the top categories and added up the number of sites on 1 June. When I came across this thread I went back and counted them again --- DMOZ is growing faster than any other directory
Ivan_Bajlo




msg:3389420
 11:12 am on Jul 9, 2007 (gmt 0)

DMOZ is shrinking because there are less and less editors each day and if that wasn't enough long time dedicated editors are getting removed for not following Meta editors party line and actually trying to fix DMOZ by removing pedophile websites which somehow managed to stay listed for years! >:-(

<Please do not republish posts from other fora. Thank you.>

If you want extreme case of how bad state of DMOZ is Slovenian language category has only 876 links at the same time Slovenian Wikipedia is closing to 50000 articles being 27th largest wiki in the World! There are probably more links on Wikipedia then in DMOZ! Directory of Slovenian websites najdi.si has 57612 links! It is interesting how wikipedia has no problem in digging up enough editors to write 50k articles while at the same time there is nobody at DMOZ?

One reason for this is that there are few metas which understand anything more then English so World categories are neglected because unwritten DMOZ rule that it is better to have dead category then risk possible spammers getting in. :-(

[edited by: Webwork at 3:03 pm (utc) on July 9, 2007]
[edit reason] WebmasterWorld TOS [/edit]

incywincy




msg:3389493
 12:48 pm on Jul 9, 2007 (gmt 0)

It's a paradox isn't it? I have submitted my core business website many times over the past months to DMOZ and it has yet to be listed.

The directory is shrinking and yet there is no shortage of newly submitted sites I'll bet.

whoisgregg




msg:3389546
 2:02 pm on Jul 9, 2007 (gmt 0)

If AOL programmers are still "working" on restoring the ODP after the crash, then I suspect they will never finish.

Webwork




msg:3389594
 3:01 pm on Jul 9, 2007 (gmt 0)

Simplisitic "ODP needs to go away/die/whatever" have no home here. I've deleted several such posts and will keep doing so. By Charter we're not the place for people to vent their frustration with the ODP. Please don't waste your time or mine on such fruitless efforts.

Please keep the conversation proactive vis-a-vis our relationship with another volunteer organization, i.e., DMOZ. If you have an issue to present please stick to an analysis of facts that can be established by reference to verifiable source material of some authority or import. The fact that anyone's site didn't get listed or that anyone's application to act as an editor was rejected is a topic that we are not going to keep revisiting or rehashing.

Thank you.

[edited by: Webwork at 4:34 pm (utc) on July 9, 2007]

cbpayne




msg:3390019
 11:35 pm on Jul 9, 2007 (gmt 0)

>I have submitted my core business website many times
>over the past months to DMOZ and it has yet to be listed.

You have answered your own question. Why did you submit more than once? How do you think any directory (DMOZ or other) should treat and site that deliberatly flouts the guidelines they agreed to when they suggested the site?

incywincy




msg:3391297
 7:06 am on Jul 11, 2007 (gmt 0)

is 3 times over the past 6 months excessive? if the site had been included already then yes i can understand that it can seen as violation. however because they are so slow in their operations it gives an impression of inefficiency and then you start to wonder if they actually even looked at your submission.

On top of that you don't even know if the editor is a competitor.

Is it any surprise that the directory is getting smaller?

cbpayne




msg:3391341
 8:53 am on Jul 11, 2007 (gmt 0)

>is 3 times over the past 6 months excessive?

Its not excessive, but is pretty silly. All you have done is put your site at a disadvantage as each submission just overwrites the previous one with the new date. There is never a need to submit more than once - any more is not helpful to you or DMOZ (and a waste of your time)

>however because they are so slow in their operations it gives an
>impression of inefficiency

Slow? DMOZ is the fastest growing directory on the www and no other directory comes remotely close to DMOZ's growth rate.

>and then you start to wonder if they
>actually even looked at your submission.

As DMOZ is not a listing service and an editor is not there to process submissions (other directories provide that service), editors are under no obligation to even use suggested sites when they work on a category. Editors can use whatever source they like for new sites. The submitted sites is the worse and most inefficient way to find new sites to lisr.

>On top of that you don't even know if the editor is a competitor.

Given that each category has greater >200 people that can edit in it, I doubt all 200 are competitors.

>Is it any surprise that the directory is getting smaller?

Have you not read the messages in this thread? DMOZ is growing faster than any other directory and is NOT getting smaller (I did post in a message above the actual numbers to show this, but in the clean up it was deleted)

g1smd




msg:3399141
 3:34 pm on Jul 19, 2007 (gmt 0)

>> DMOZ is shrinking because there are less and less editors each day <<

Many hundreds of new editors are approved each month.

Less editors?

No.

It must be at least a year since I last looked in this forum.
Shame to see the conversation is still the same as it was in 2002.

hutcheson




msg:3399182
 4:02 pm on Jul 19, 2007 (gmt 0)

Now, G1, be fair: the people who are prognosticating death today, may not have known how to use a keyboard in 2002. Any community is always picking up new members who, not knowing history, simply re-enact it. The babes in mothers' arms of today may well be starting their own chant of the same refrain in 2020 or so.

martinibuster




msg:3399210
 4:44 pm on Jul 19, 2007 (gmt 0)

hutcheson, the OP is not prognosticating the death of the ODP, and it's unfair and a bit of an overreaction to characterize it as that. The OP has made an observation of a phenomenom in one section of the ODP.

Interesting. Let's discuss. ;)

One of the reasons some cats are shrinking is because some editors only include their own sites, or those of their associates, and have shut out their competitors, even larger more reputatable sites.

g1smd




msg:3399216
 4:53 pm on Jul 19, 2007 (gmt 0)

Even if "some editors" did do that, there are more then 200 other editors that could edit in any given category and so you would never know that one editor wasn't listing some stuff there.

martinibuster




msg:3399245
 5:29 pm on Jul 19, 2007 (gmt 0)

you would never know that one editor wasn't listing some stuff there.

I in fact know that not only are some editors not listing competitors, they are only listing their own sites, and those of their associates.

there are more then 200 other editors that could edit in any given category...

200 editors cannot make a meaningful impact. Given the size of the ODP, any reasonable person can understand that.

Look, I'm not saying the ODP is riddled with self-interested editors. It is not. I'm only pointing out one reason why some categories are not growing.

theBear




msg:3399380
 7:53 pm on Jul 19, 2007 (gmt 0)

simonmc,

Back many years ago there were two musts for any site:

1: Getting listed in DMOZ aka ODP

and

2: Getting listed in Yahoo's directory

I'm willing to bet that most people who have done any searching have used the results of DMOZ, very likely without even knowing it.

DMOZ has been abused many times and has had its share of problems. But it still is worthwhile to have a DMOZ listing.

I happen to know several DMOZ editors and just so you know there is at least one participating in this thread.

theBear




msg:3399383
 8:00 pm on Jul 19, 2007 (gmt 0)

In addition looking at the rdf dump sizes the ODP has been growing overall.

So while one category may be shrinking it is not true of the overall directory.

It is also possible that sites have been reclassified as well.

Plenty of reason for the observed changes outside of and as well as the one being discussed.

rogerd




msg:3399418
 8:35 pm on Jul 19, 2007 (gmt 0)

Perhaps a more accurate gauge than "growing vs. shrinking" is whether the number of listings (and perhaps number of editors) is growing in proportion to the number of active websites. If DMOZ is growing its absolute number of listings, but if that represents an ever-smaller percentage of active sites, then one might say DMOZ has a problem.

Anyone have those stats?

System
redhat



msg:3399615
 9:46 pm on Jul 19, 2007 (gmt 0)

The following 2 messages were cut out to new thread by encyclo. New thread at: directories/3399613.htm [webmasterworld.com]
8:42 pm on July 19, 2007 (utc -4)

blend27




msg:3399563
 11:23 pm on Jul 19, 2007 (gmt 0)

--- any more is not helpful to you or DMOZ (and a waste of your time) ---

cbpayne, you right about that one.

In the past 2 years we had only 2 major competitors rise to the level of an authority, one died in progress.

Google Shows 17 sites for their version, that is another story.
There are 9 in DMOZ in a category in question(including 1 dead site and one that is aqua color- you know the old school type)

no editor for almost 2 years

I have a site that is based in US and has a most unique product set at any given time for the past 4 years, any of my competitors will tell you that.

The only problem I came across(suggested 2 times so far) is that the editor gets no access to the site due to the fact they are trying to access my site from an IP range that is blocked by firewall due to some A.W. running a bot from an IP range with in, or the user agent string contains a specific set of characters, and in reality could be identified as a spider(partially harvester, 'JAVA' for example).

So, god bless

g1smd




msg:3399583
 11:59 pm on Jul 19, 2007 (gmt 0)

>> DMOZ is getting smaller <<

Patently untrue. Time to close this thread?

callivert




msg:3399585
 12:03 am on Jul 20, 2007 (gmt 0)

None of the DMOZ supporters or editors who have made appearances on this thread have addressed the original post.
The claim was very specific. One category- main shopping- is shrinking, despite the fact that this is a large and important category. Is this true? We keep getting unsourced "word of mouth" stats about how DMOZ as a whole is expanding, which doesn't address the specific claim of the OP.
If main shopping is static, then one or both of the following factors may be at work.

1) insufficient editors (of that category, not DMOZ in general)
2) corrupt editors (of that category). Excuse my bluntness, but I'm just stating outright what others are insinuating- and I'm only saying that it's a possibility.

Both of these possibilities are concerns - valid or otherwise - held by many about DMOZ in general. That's why the issue is interesting, as this may be concrete evidence of more widespread problems.

BillyS




msg:3399588
 12:10 am on Jul 20, 2007 (gmt 0)

My category got smaller too. About a year ago I reported six or seven (about 20%) dead links or redirects and they quickly removed those sites. Editors quickly removed those sites.

Since that time not one site has been added to DMOz in that category. Of course g1smd probably has good reason to dispute such as observation since this is only one data point.

SEOPTI




msg:3399601
 12:26 am on Jul 20, 2007 (gmt 0)

Their branding is shrinking.

hutcheson




msg:3399602
 12:26 am on Jul 20, 2007 (gmt 0)

>I have a site that is based in US and has a most unique product set at any given time for the past 4 years.

Back in the last millenium, we had some really bad experiences with sites that artificially generated "unique product sets." (Experienced editors will still reach for a blunt instrument if they hear the word "Vstore.")

Basically, for us, "product set" is not something whose uniqueness can have any conceivable significance.

austtr




msg:3399626
 1:03 am on Jul 20, 2007 (gmt 0)

Why do we continue to rehash the DMOZ saga over and over again, with the inevitable angst that always seems to accompany such discussions?

Because its free and the belief that Google rewards sites that have been able to make it into DMOZ.... right? Remove Google from the equation and there would be the same number as posts as we see for BOTW, Skaffe, JoeAnt, Ezilon and all the other global directories.... ie... virtually none.

Why not challenge that assumption and try to establish if Google does in fact reward DMOZ listed sites? In my experience sites not in DMOZ get the same amount of traffic as similar sites not listed in DMOZ. The DMOZ link is just a link like any other from a PR0 page.

I know that in my area (regional) the number of DMOZ listed sites have not increased in years and those that are still there fall a long way short of being a selection of the best available. Similar scenarios apply to other regional cats.

But that is a quality control issue for DMOZ and if they don't address it, it is inevitable that parts of DMOZ will become increasingly irrelevant.... just like any other under populated and out of date directory.

Why would Google continue to give extra credibility to DMOZ listings if that continues to be the case?


theBear




msg:3399636
 1:22 am on Jul 20, 2007 (gmt 0)

austtr,

"Because its free and the belief that Google rewards sites that have been able to make it into DMOZ.... right?"

Nope, you should ask yourself why Amazon, L.L. Bean, and all the rest are there.

The name of the game is exposure, that it is free is just icing on the cake.

Marketing 101 at its best.

theBear




msg:3399679
 3:16 am on Jul 20, 2007 (gmt 0)

"One category- main shopping- is shrinking, despite the fact that this is a large and important category."

Yes that category is shrinking, is that in and of itself important or is that category even important?

Well that is debatable.

Let's see you take the old list and remove all of the sites that have gone belly up, add in the stricter rules about what gets listed and is it really a surprise that one of the more abused (from whatever direction) categories shrinks.

BTW, I don't give a hoot one way or the other about this topic.

I just think you need to look at it from more than the viewpoint of I have a site and .... therefor ....

If the category shrinkage is important but DMOZ isn't what is one to think, that DMOZ isn't important but why would a site owner want a listing if it isn't important (just don't attempt explaining that to a marketing type).

This 103 message thread spans 4 pages: 103 ( [1] 2 3 4 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Directories
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved