Forum Moderators: open

Message Too Old, No Replies

Is Google hurting the ODP?

         

bcushion

3:09 pm on Jan 15, 2003 (gmt 0)

10+ Year Member



It seems that everyone these days is having problems getting into ODP and many are talking about it. It also seems that, except for Google, this directory has little to offer. I may not have a good historical perspective but could this ODP "decendancy" come as a result of the Google ascendancy and Google's utilization of the ODP? The lack of volunteer editors; assumptions that a submitted site can't get in because editors are too busy (rather than a competitive rebuff); "gold rush mentality" purchases of expired domains existing on ODP; heavily spammed categories and other signs seem to point to the fact that SOMETHING made ODP of such supreme importance that it lured into its fold the very kind of people that may eventually destroy it.

I know ODP was heavily influenced from the beginning by volunteer editors eager to place their own sites but it seemed these inexperienced and eager people just added to the excitement. It seems now the sway of the top level editors has changed the entire character of this once dynamic project. This has held true in commercial categories I followed. My own self interest caused me to join, leave and now disregard this once important directory.

I don't think ODP can continue in any importance without significant COMMERCIAL categories and the spamming involved certainly makes them insignificant. Google's emphasis on this directory is still strong, attracting and creating opportunity for ever newer spamming techniques to ODP and I wonder, when Google casts it aside, if it will soon die.

WindSun

6:24 pm on Jan 24, 2003 (gmt 0)

10+ Year Member



"To the person who is looking for the content that you would have us exclude,.."

I never stated what what I "would have you exclude".

However, the fact is, there is absolutely NO quality control or quality ratings in ODP. That makes it very difficult to actually use it as a directory directly. You HAVE to rely on the Google filtered and ranked results for most categories.

For example, I just now pulled up a forum site at random from the computer gaming area. Their is practically nothing on the site ("coming soon1"), and the last forum entry is: "News: 10th of August 2001".
There are 200+ entries in that category. Far too many of them that I checked have such useful pages as:

"Mark Entertainment was closed on 00:00 April 16th, 2001"

Another one, last updated 1996, that consists of a single page with 6 links, 5 of which are dead.

Another one: "For the time being, I've removed this page from my website." (dated Feb 2001).

Or: "Hey everybody... welcome to my old, old website..."

Etc. etc. etc....

There ARE some useful sites amongst all that mess, but actually finding them took some work. In that section alone, it appears that nearly 50% of the sites are dead or abandoned.

Now THAT is the kind of stuff that should be excluded...

rfgdxm1

8:18 pm on Jan 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>Submitters are not our customers, they are voluntary helpers. Webmasters are not our customers. We are yours. And all that you say about listening to your customers: if you want to be listed by your customer, you'd better be listening!

Heh. Webmasters often see it backwards. Not long ago, as an ODP editor I reviewed a site, and promptly listed it within 2 days after it was submitted. However, the webmaster quickly sent me feedback complaining that I had altered his keyword stuffed submission, and submitted it again actually mentioning "keywords added". I am quite serious this happened. Needless to say this I didn't keep this webmaster "customer" satisfied, and dropped the update request in the bit bucket and left the description as it was. ;)

mosley700

1:30 am on Jan 28, 2003 (gmt 0)

10+ Year Member



hutcheson
Do you volunteer your time at the ODP for your own reasons? Do you not have anything more valuable to do with your time? By volunteering your time, are you helping starving children?

>>SURFERS are OUR customers.<<

I'm a surfer. Most people who help build the directory, or the people who bring problems to your attention, are surfers. AND webmasters. You have such a strong ANTI-webmaster attitude it's amazing you list any sites. Or do you? Or do you just delete sites? Most people on the Internet don't know what dmoz is. It's webmasters who built the Open Directory. It's webmasters who keep it alive. It's webmasters who submit their sites that hwelp build the directory. It's webmasters who volunteer their time to build the directory that make the directory a half-decent place to find sites. Where would the ODP be without thousands of webmasters who helped build the directory? Nowhere.

WindSun

6:25 am on Jan 28, 2003 (gmt 0)

10+ Year Member



"Our concern is for the person looking for content..."

That no doubt explains why there are 2,930 listings for Epinions...

kfander

6:52 pm on Jan 28, 2003 (gmt 0)

10+ Year Member



>> It's webmasters who built the Open Directory. <<

Wrong.

>> It's webmasters who keep it alive. <<

Wrong again.

>> It's webmasters who submit their sites <<

Wrong again. I expect that most of the really useful sites were listed by editors who found them on their own.

>> It's webmasters who volunteer their time to build the directory that make the directory a half-decent place to find sites. <<

I haven't seen any stats, if indeed there are any, but I would guess that most ODP editors are not webmasters, except perhaps of a personal page.

Other than the fact that webmasters build the sites that we list, the ODP has not been at all dependent upon submissions.

When you are accepted as an editor for a category, it is expected that you will go out and find sites to add to the category that you are maintaining. In some ways and in some categories, submitted sites interfere with this process.

[edited by: kfander at 7:20 pm (utc) on Jan. 28, 2003]

hutcheson

7:05 pm on Jan 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



mosley, editors are REQUIRED to have SOME motive OTHER than maximizing a particular webmaster's income. It's not opposition so much as impassive indifference.

The ODP lists websites, not webmasters. This may seem like a subtle distinction, but think of it this way: so long as the website remains useful, the sudden death of the webmaster will have no effect on the ODP. But even if the webmaster remains active in society and business, the sudden removal of the website will instigate a change in the ODP listing.

This pattern of behavior doesn't reflect a philosophical statement about the futility of human existance; it is merely recognizing the limits of the web-directory medium.

Which, now that you mention it, is actually relevant to the topic at hand. The ODP can't do everything: its only justification is that it does _something_ well enough for it to be worth doing: where "well enough" necessarily entails "better than any other approach using no more resources".

Another way of asking the question is: of the things the ODP does better than Google could have done without the ODP, are any of them worth doing (at least, to the people doing them)

If/when the answer becomes "no", it will be fair to say Google killed the ODP. But note, there are at least two ways of answering "yes": 1) The ODP does this better than Google; 2) If Google did not use the ODP, then we'd have to recreate the ODP to get this done. And many Google enhancements promote #2 even as they undercut #1.

mosley700

9:32 am on Jan 29, 2003 (gmt 0)

10+ Year Member



hutcheson,
I spend hours every week in [directory.google.com...] and the subcats.
That URL would not exist w/o the ODP. Neither would it exist w/o Google. I like to see the sites in categories, and with PageRank next to the URL.
It's my opinion that Google is making the ODP what it is. I never would want to search the ODP without PageRank assigned. Without Google, the ODP would be another web directory, of which hundreds exist.
The ODP is the best web directory we have; is that to say it cannot be improved?

@kfander,
>>Wrong.<<

>>Wrong again.<<

>>I haven't seen any stats, if indeed there are any, but I would guess that most ODP editors are not webmasters, except perhaps of a personal page.<<

What is this? Assume-a-rama? Of the 30 or so ODP editors I know, only 2 (?) or 3 are not webmasters or SEO, or website administrators.

I know it's fun to "puff the magic dragon", but we're having an adult conversation here.

And, most websites are submitted. Most editors have so many unrevs they don't have time to go out looking for good websites.

kfander

2:03 pm on Jan 29, 2003 (gmt 0)

10+ Year Member



>> Of the 30 or so ODP editors I know, only 2 (?) or 3 are not webmasters or SEO, or website administrators. <<

I think that's called anecdotal evidence, and it is less reliable than the assumptions of someone who has been an ODP editor for a few years now.

During the 2000 elections, I found it difficult to believe that Algore almost won the election, and certainly that he might have won the popular vote; for, you see, I knew very few people who voted for him.

Similarly, it was hard for me to believe that Clinton was elected to two terms, since almost no one I knew voted for the guy.

Most of the people I know are Baptists, which leaves me wondering how all of those other churches manage to pay the bills.

Do you get it? You are a webmaster and you hang out at places like Webmasterworld, so it can be expected that the majority of the ODP editors you're familiar with are webmasters.

>> but we're having an adult conversation here <<

... But in adult conversations, we try to separate anecdotal evidence from the real stuff, or at least recognize it as such, as I did when I said:

>> I haven't seen any stats, if indeed there are any, but I would guess that most ODP editors are not webmasters, except perhaps of a personal page. <<

In fact, we may even define the term "professional webmaster" differently. To me, it refers to someone who depends on the websites that he creates for a living, at least in part. Yet often I see that anyone who has a website is identified as a professional webmaster, particularly when that person is an ODP editor and the person making the reference is interested in making inferences.

Dynamoo

4:18 pm on Jan 29, 2003 (gmt 0)

10+ Year Member



However, the fact is, there is absolutely NO quality control or quality ratings in ODP. That makes it very difficult to actually use it as a directory directly. You HAVE to rely on the Google filtered and ranked results for most categories.

Oh but there *IS* - it's peer review and meta editors.

But the ODP isn't a competitor to Google. If we're talking about site descriptions the problem is that writing a one or two sentence summary of a site is extremely difficult, even for experienced editors. It's impossible to summarize everything in a site AND keep it up to date as the site changes. And since most submitters write sucky descriptions, it's usually the editor who has to do the job for them.

There is absolutely no doubt that Google adds value to the directory listings - one by providing a comprehensive full-text search facility of the sites, and the other by sorting them into PageRank to help identify authorititive sites. The ONLY ranking that an ODP editor can give is to mark the site "Cool".

But probably the key use of the directory is NOT whatever the title says BUT the process of cataloging and categorising sites, so that you can actually FIND a list of (say) Cobol Compilers and categorise them together. Try doing that *without* a categorised directory and you won't get very far very quickly.

Back in the early days of the web when I had to teach people how to actually find stuff, I used to tell them that there were TWO important tools to use - a search engine (i.e. AltaVista) and a directory (i.e. Yahoo) and I used the analogy that a SE was like looking something up in the index of a book, and a directory was like looking in the Table of Contents. Now, Google is about the ONLY SE that combines the two in any meaningful way. Hats off to Google for being so clever.

Look, lots of things about the ODP could be better. The broken RDF dump *is* a nightmare. There are *never* enough good editors to look after it all. Sometimes there *is* abuse... but all the other directories are almost dead or actively committing commercial suicide (Y!, LS) so pretty soon the ODP is going to be about the only directory worth anything.

hutcheson

4:43 pm on Jan 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>It's my opinion that Google is making the ODP what it is.
I don't think so. Google is recognizing one of the things the ODP is, and Google's recognition is (justly) contagious. But the ODP was there for other users (remember Lycos? Netscape portal?) before the rise of Google. You could with equal justice have said that Google is suppressing use of the ODP by other portals. (Without Google, there would probably be other portals and search engines, some of whom would almost certainly have used the ODP.)

>I never would want to search the ODP without PageRank assigned.
I think we can probably all agree here: the Google engineers have found multiple cool ways of adding value to the ODP content. (You probably know that ODP directory editor pages have a link to the corresponding Google page, for the convenience of editors checking pagerank.)

>Without Google, the ODP would be another web directory, of which hundreds exist.
>The ODP is the best web directory we have;

A bit of a contradiction here. I think the first is the error. Without Google, the Open Directory would still be the directory of the web's largest captive-audience (AOL), the world's largest hierarchical taxonomy, the largest internet directory, and and -- by an order of magnitude -- the world's largest free directory.

>is that to say it cannot be improved?
I don't think we'll find any disagreement here. Editors have their own little lists of improvements that could be made.

There are several different issues here. One is "improvement" (which editors are supposed to do); another is "distribution and added value" (which Google and to a lesser extent Teoma do); still another is "replacement" (which Inktomi might do, if its results without the ODP were so good that nobody felt the need for a web directory any more, or which Yahoo might do, if its organization proved significantly more efficient at building a directory valuable to surfers).

WindSun

6:23 pm on Jan 29, 2003 (gmt 0)

10+ Year Member



"And since most submitters write sucky descriptions, it's usually the editor who has to do the job for them..."

You mean like these outstanding examples of editorship, taken just a few minutes ago from ODP. Note that most examples are the FULL description given in ODP:

"A divulgative site"

"An article from Nature.com."

"A short article in New Scientist without much new information."

"A one stop site for all things soul-related."

"...A superb article exploring the horrible parallels between slavery and animal abuse..." [no editorial comment here ;p)

"Helps Horses to the life they deserve."

"Imagine reaching for an apple on a tree and having your hand suddenly impaled by a hook that yanks you into an environment where you cannot breathe. This is what a fish faces when hooked. While fishing may seem fun to some people, it is important to remember that on the other end of the line there is a terrified animal fighting for his or her life. These fascinating animals suffer and feel pain just as all animals do."

"Milk production is bad for the environment, for animal welfare, and for human health, this PETA site explains."

"Welcome to a rapidly growing backlash against the Dairy Industry, its disinformation for profit, and the hundreds of millions of dollars it spends to perpetuate the myth that a liquid from another species is 'nature's perfect food' for humans... Ordinary dairy milk is a deadly poison. Each sip contains growth hormones, fat, cholesterol, allergenic proteins, blood, pus, antibiotic, bacteria and viruses!"

"These Warriors for Justice need Your Help!"

"The story of environmental racism in Arizona, where Hopi and Navajo people are to be forcibly relocated to a uranium dump in the year 2002."

"There is an awful lot unknown about hazards of new [GM food] crops and until it is fully tested we should not be subjecting people to risks, least of all young children."

Etc. etc...

What concerns me about many of the listings is not the bad grammar or poor spelling, but the editorial bias in many of them.

mosley700

11:17 pm on Jan 29, 2003 (gmt 0)

10+ Year Member



Hutcheson,
May I commend you on your willingness to discuss these issues? And I used to think you wore blinders that said "ODP is Perfect" on 'em.
;)

>>Without Google, the ODP would be another web directory, of which hundreds exist.<<

I said that meaning, it is impossible to search the ODP and find what I'm looking for. Google's search algorithim enables users to search the directory and find the sites and the categories I'm looking for.

And doesn't it make more people willing to help out in the ODP? If I want to ensure that my sites get listed in Google, one sure way is to submit to the ODP. In my opinion, it helps motivate volunteers to apply at the ODP.

Regards.

rfgdxm1

5:35 am on Jan 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>Do you get it? You are a webmaster and you hang out at places like Webmasterworld, so it can be expected that the majority of the ODP editors you're familiar with are webmasters.

My guess is most are. Doesn't necessarily mean anything bad. Most ODP cats are non-commercial, and webmasters of non-commercial site typically wouldn't abuse the ODP. In fact, they may be exchanging links with other sites in the cat theirs is.

rfgdxm1

5:40 am on Jan 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



One comment WindSun. Editorial standards have changed at the ODP over time. Those many have been from years ago. Also, every now and then an editor just isn't on the ball and lets something slip by that shouldn't have been.

rfgdxm1

5:48 am on Jan 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>I said that meaning, it is impossible to search the ODP and find what I'm looking for. Google's search algorithim enables users to search the directory and find the sites and the categories I'm looking for.

No, you can do this with the ODP. The ODP search is designed to find relevant cats, not specific sites. Thus, if you want to buy a widget look for a cat with widget sellers. I also just found one of my sites in the ODP by searching on a word that no other site in the ODP uses. I found my site quick.

WindSun

3:37 pm on Jan 30, 2003 (gmt 0)

10+ Year Member



"Similarly, it was hard for me to believe that Clinton was elected to two terms, since almost no one I knew voted for the guy..."

What you have to realize is that some people voted for him several times. ;p

"Most ODP cats are non-commercial, and webmasters of non-commercial site typically wouldn't abuse the ODP.."

Well, yes and no.

Most categories are non commercial, but for the most part the LARGEST cats are commercial.

And as for non-commercial webmasters not abusing.. well I recall some pretty blatant abuses a couple years ago by a certain religious group. When I was editing I found out that some types of "true believers" in a cause can be just as bad or worse than your typical affiliate monger.

rafalk

7:08 pm on Jan 30, 2003 (gmt 0)

10+ Year Member



When I was editing I found out that some types of "true believers" in a cause can be just as bad or worse than your typical affiliate monger.

This is absolutely true.

rfgdxm1

8:43 pm on Jan 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There of course will be cases where editors of non-commercial cats abuse. Likely more common in commercial cats simply because since there is money on the line those will tend to attract more editors with bad motives. I can see why in certain cases such as religion sites "true believers" may engage in abuse. One reason this may happen is that when it comes to commercial sites, as an editor metas likely won't buy the argument a site just "sucked", and that is why it wasn't added, unless the editor can justify this well. The problem with dealing with abuse from "true believers" is that they have to be spotted as being such, rather than neutral but with high standards.

WindSun

2:05 am on Jan 31, 2003 (gmt 0)

10+ Year Member



"rather than neutral but with high standards..."

True believers are much harder to spot, and can hang around for years unless someone else with the same but opposing interests comes along.
And while you usually tend to think of religious areas as being the main target for these people, they have also been spotted in such diverse areas as environmental, UFO's, and conspiracies.

shangri_la

1:11 pm on Feb 5, 2003 (gmt 0)

10+ Year Member



ODP editors have been reviewing/adding thousands of sites since the last sucessful RDF dump in September, 2002. I personally have reviewed hundreds of sites in Web design, Shopping, Regional, and more recently Recreation/Equestrian - that did not show up until the recent RDF temporary dump.
Sites are being added, it's just the RDF dump that is not allowing them to be displayed on the pu8blic pages, or in the search results. As soon as that is perminatly fixed, you will see faster and more accurate results from the ODP. Technical progress is sometimes hard and has many bumps on the way, but in the end, upgrading the system will only benifit the end user. Stop complaining, and volunteer to be an editor today!

Bluesplinter

2:21 pm on Feb 5, 2003 (gmt 0)

10+ Year Member



it's just the RDF dump that is not allowing them to be displayed on the pu8blic pages

No, the sites you add show up on the public pages as soon as that page is regenerated (anywhere from instantly to several minutes later, or longer depending on how backed up it is). It has nothing to do with search or the RDF.

shangri_la

9:50 pm on Feb 5, 2003 (gmt 0)

10+ Year Member



Now they do! .. but they weren't showing up before. I had a quiery before xmas from a submitter whose site I had added, but it wasn't showing up on the public page, even afer several refreshes. I checked the next day, and it was still the same. That was before though, not now. The data seems nice and fresh now.
Yesterday I checked my public page for one of my categories, and it stated that the page hadn't been updated since January 25th, 2003, and yet, I had edited that category practually every day since that date.

Anyway, my point was, that when there are upgrades in process, to just be patient.

jp29997

6:09 pm on Feb 6, 2003 (gmt 0)

10+ Year Member



this was a great thread, I'm really sorry to hit it so late but...I can answer the big question! Is Google hurting the ODP? The answer is yes. I'd prefer the question rephrased to "Is spam motivated by an increase in PageRank hurting the ODP?" The answer to that is a resounding YES. When you edit a commercial category you have to go in every couple of days, grit your teeth and go after the unreviewed sites. Unfortunately, you don't get to just delete the inappropriate submissions willy-nilly, you can delete duplicates, you can delete affiliates but anything else, you are supposed to find the right directory to move it to--which is really much more difficult than it sounds, not surprisingly, the world's largest human-edited directory has a very complex taxonomy. I count my monolingual self very lucky if I can even tell what language a submission is in. It's an enormous amount of work, and dmoz has been very proactive in efforts to handle the workload. But, this is one of those arguments about the negative aspects of Google that boil down to the fact that if anyone had known that Google was going to dominate it's market so thoroughly, things might have been done differently. A carefully edited spam-free aggregation of links really helps Google, because it gives them something to tune their results against and it is an easy way to inject relevancy into their algorithm. It's just that the submissions that Google generates because of the influence they allow ODP to have on PageRank are overwhelming to a volunteer staff.

sorry for length

rfgdxm1

9:00 pm on Feb 6, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've never seen a commercial cat that tends to attract spam from the editors side? How bad is it jp29997? While I edit a largish cat space (around 5,000 sites), none of it is the sort that tends to attract spam. Most of the spam I see, which ain't much, comes from the carpet bombers who try submitting to every ODP cat.

gingaaar

9:55 pm on Feb 9, 2003 (gmt 0)

10+ Year Member



My God, I cant read all this, But I must say that if they do actually find time to find a suitable category for that 'Spam' website... lets hope its mine!

:))

This 115 message thread spans 4 pages: 115