Why Does Google use DMOZ? Gary Illyes: "I have absolutely no clue"

[engine]

Google was asked why it used DMOZ at all, and Google's Gary Illyes responded with "I have absolutely no clue"
[twitter.com...]

Could it be one of those things that was a good idea at the time, and then it became forgotten!

noodp can often be your friend.

[RedBar]

Talk about a blast from the past therefore I decided to have a look and saw "In Partnership With AOL" ... What's that all about? ...Off to Wikipedia to check it out:-)

[RedBar]

I think I must have been asleep when AOL became "involved"!

[engine]

AOL was many, many, years ago when it acquired Netscape.

[rainborick]

I know why Google relied on DMOZ historically. It was because the entries were curated by editors and so the titles and descriptions tended to be far less SPAMmy than what some webmasters used and they tended to remain unchanged. And, of course, the directory itself was highly regarded ages ago. But that time has past, and I don't think its worth rehashing its history.

[JS_Harris]

At one time Google liked the idea of DMOZ so much, or perhaps feared Yahoo's version of the directory, that they created their own copy of it. This was back in 2000 I believe. 10 years later Google shut their version down and made all such directories irrelevant to their core business.

That's all you need to know in terms of DMOZ relevance. The noodp tag is probably a waste, I believe noodp is now the default behavior.

[Robert Charlton]

The noodp tag is probably a waste, I believe noodp is now the default behavior.

I wouldn't bet on it. There are occasional CMSs or site configurations that didn't plan for description elements, and I've seen sites built on these return ODP/DMOZ descriptions... not long ago, if I remember correctly.

[martinibuster]

Taxonomy.

I'll buy a beer to the first person who names at least one reason why the DMOZ Taxonomy is of interest.

[tangor]

Uh... it's a place to start (building an index)? (ducking)

[Robert Charlton]

Taxonomy is of interest

Wild guess as to what specifically you're after, mb... as you've always got something very precise in mind... but, and this is a quick and maybe sloppy guess....

ODP was probably (following up the Yahoo Directory, which was limited in scope), the best human curated semi-structured data set that Google had to use as a seed set for topically organizing the more popular segment of its own network. I've noted DMOZ in posts about site structure as a prime example of a hierarchically organized taxonomy.

It could have been in effect the core set from which Google built its initial web graph. Google overlayed that directory with layers of metrics, and also added into its own database topical areas like medical sites selected by medical experts. I think that ODP, as Yahoo, may have had a lot to do with Google's initial quality. Even if PageRank by itself might have ordered sites by popularity and citation value... Yahoo and ODP provided an initial calibration of topicality of reasonably good sites... and Google is now continuing to refine as they've broken that down into n-grams, analyzed observed linguistic patterns, identified entities and attributes, applied vector analysis, and now user behavioral analysis and machine learning.

As I said, this is rough... but Yahoo and ODP come as early sets of data that could have proved to be very useful. Wikipedia is probably now more useful.

[Andy Langton]

You can still download the entire hierarchy ;)

[rdf.dmoz.org...]

[martinibuster]

a prime example of a hierarchically organized taxonomy. 

Bingo. DMOZ is a convenient taxonomy of the web, broken down by topics and sub-topics. A great way to form topical buckets for an algorithm to classify pages or sites into. There are algorithms today that use Wikipedia in a similar manner but for different purposes.

Gary Illyes: "I have absolutely no clue"

He's spot on.

[edited by: martinibuster at 1:58 pm (utc) on Mar 15, 2016]

[randle]

DMOZ & Google?

DMOZ was the dedicated wife that helped put the struggling doctor Google through medical school. Once the Doc graduated and was making oodles of money he immediately dumped the wife for the hot looking nurse and never looked back.

I'm not sure there would be a Google if it weren't for DMOZ, or certainly wouldn't be what it is today at this point in time.

[RedBar]

I'm not sure there would be a Google if it weren't for DMOZ, or certainly wouldn't be what it is today at this point in time.

Thinking back through the hazy memories of time, especially the all-nighters in the late 90s and early 00s, I seem to remember Google did use DMOZ a lot...tell me my memory's not that bad!

[EditorialGuy]

I'm not sure there would be a Google if it weren't for DMOZ, or certainly wouldn't be what it is today at this point in time.

Couldn't they have used Yahoo! just as easily? (I imagined they used both.)

Uh... it's a place to start (building an index)? (ducking)

Yes, and now that they have an index, they can be just like everyone else and ignore DMOZ. (Crappy directories are soooooo 1990s.)

[randle]

Couldn't they have used Yahoo! just as easily? (I imagined they used both.)

DMOZ would periodically do a dump of their data which Google would utilize as a basis for their results.

All kidding aside, Google was going to be Google eventually, but the treatment they gave DMOZ was less than admirable once they started taking off.

[martinibuster]

I'm not sure what treatment by Google you're referring to.

It was AOL that purchased DMOZ then kept it on a bare bones scale. In 2003 I started a thread about this issue, suggesting that DMOZ would be better off if it were allowed to become an independent non-profit organization, with a board of directors, a development director to raise funds, etcetera. As far back as 2003 it was evident to me that DMOZ was on a downward trajectory and was in need of stewardship with direction.

DMOZ was pretty much neglected by AOL and allowed to wither into irrelevance rather than evolve with the times like a normal organization.

It's on AOL, not Google.

[randle]

It was AOL that purchased DMOZ

Perhaps if there was another suitor that stepped up to the plate.....

[martinibuster]

DMOZ came with a stipulation, as I recall, that whoever purchased it had to maintain DMOZ as a free and open source project. It couldn't be monetized.

In any case, DMOZ may not necessarily need a suitor or buyer. Archive.org and Wikipedia are independent organizations that rely on donations. DMOZ could have gone that route with proper leadership. Is it too late?

[randle]

I won't lie, I miss the days when DMOZ was a real factor in search, theres never really been anything like it.

My hazy recollection of the situation a hundred years ago was DMOZ asked Google for a fairly sizable donation to keep it going and Google flat out declined. I have always felt they had some type of obligation to the entity, (a non-profit no less) that played such a pivotal role in their emergence. The two were joined at the hip for a while there. You don't drop your old friends that stood by you when things get really good for you, and not so good for them.

Is it to late? I don't know but I would sure love to see it come back to relevance.

Of course take all this with a grain of salt, I'll admit I'm probably turning into a nostalgic internet old fart. I was a lot happier person before Panda, Penguin, the rise of big brands, the decline of key word rich domains, 4 ads at the top of the results page and a host of other changes that always seem to make me less money and Google more even though they keep saying these changes are awesome advances that will improve the user experience and make everyone super happy.

Well, I gotta go - theres an album skipping on my turn table.