Welcome to WebmasterWorld Guest from 220.127.116.11
Moderator's Note: I've split this thread off from the existing "the DMOZ server is down" [webmasterworld.com] thread as I believe the continuing delay raises issues that merit airing, including what the lack of support may portend for the ODP and what, if anything, ought to take place.
This is NOT an invitation to speak poorly of the ODP. Like it or not the ODP is a volunteer enterprise and it is that volunteerism, in service of the project, that entitles the ODP and its numerous good faith editors to respect in my book. This problem is not of their making so "the ODP" - which IS the volunteers - ought not to take the brunt of any criticism for this lingering situation.
I stay to my opinion DMOZ is not what they intended to be. As an "authoritive" directory, to be down for more then 1 month now (6 weeks? I've been checking it daily since October 20 , I think) is really lame. And I don't count here broken links or inapropriate listings, nor submissions that takes years to shows up...really poor resource for Google
[edited by: Webwork at 1:35 am (utc) on Nov. 29, 2006]
Legalities, technical, philosophical reasons? Hey, just curious. If ODP needs a bake sale to fix or buy a box there are probably about 50 million websites out there that owe them a favor.
With Google (and others) using ODP as the source of their own directories, would it really be "inappropriate" for Google to maybe throw a bit of support their way? I mean Google's buying up acreage of server space . . . they can't (or shouldn't) give ODP a hand?
dmoz is owned by Netscape/AOL, why should/would Google fund them?
As time passes this is starting to look more like "Oh, please sir. May we have another server in a timely manner?" The gruel is getting a bit thin as time passes. The master is looking a bit callously indifferent to the situation.
Now, I know that DMOZ isn't a profit center for AOL/Netscape but this "abandoned child" approach to the situation would be a bit like McDonald's not paying the electric bill for any Ronald McDonald house it underwrites. By comparison, if the AOL email servers went down how long would it be before replacements were racked up?
So long as Netscape still has its link at the bottom of every page and every page also has a "In Partnership with AOL Search" logo at the top Netscape and AOL's collective corporate consciousness needs to wake up to the fact that they are letting down a very large number of folks.
The ever expanding delay is just wrong and the merits of the many volunteer editors considerable contribution needs to be acknowledged by Netscape and AOL by the corporations getting on the ball.
So say'eth Webwork, in this brief editorial moment.
[edited by: Webwork at 1:47 am (utc) on Nov. 29, 2006]
Its the editor server - which includes all editing, editor application, website listing suggestion and listing update functionality - which is down.
I'm not able to predict by when it will be up again.
The ever expanding delay is just wrong
As I understand it, the problem was caused by a set of events which could be described as a "perfect storm" - i.e. a combination of factors, each of which, taken individually, wouldn't have been anywhere near as serious.
I've dealt with many serious server and network failures over the years. I've had several phone calls or pages at breakfast, and headed straight to the airport to fly hundreds of miles to deal with a crisis like this one.
If ODP/DMOZ say the situation is "still being worked on" then I think we can assume they know what they're talking about. We've all waited X weeks, what's so wrong with waiting X+1 weeks?
If they bring the systems back online without a resilient backup which is fully-working (and fully-tested) then they are asking for lightening to strike twice.
As time passes this is starting to look more like "Oh, please sir. May we have another server in a timely manner?"
I can say from bitter experience that servers and hosting are cheap compared to the time and effort expended doing data recovery and building a resilient system so your system doesn't go down in the same manner next time your hardware fails.
Its the editor server - which includes all editing, editor application, website listing suggestion....
If shortage of quality/genuine volunteers is already a problem isn't this a more important server - taking the long term view - than all those others?
reducing bandwidth and server space in their basement that could easily be re-deployed to the step child ODP that is IMHO just as important or more as the mother house
This is not about servers or bandwidth.
If you've ever sat down to design a resilient system, you'll understand that throwing servers and/or bandwidth only solves part of the problem.
In a nutshell: servers and bandwidth are cheap. "High availability" is NOT
I could (and would) donate spare servers and bandwidth to ODP tomorrow - but I doubt it would help, so I'm not even going to offer.
the long term view
IMHO the long-term view IMHO is that losing a few editors is fine, as long as the sysadmins concentrate on coming up with infrastructure that is resilient.
You can always recruit new editors. You don't lose all your data (twice!) and recover from this as easily...
It doesnt matter if you are for or against this directory the fact is from a business point of view it doesnt provide any financial return what so ever - Its a complete liability.
So from this point of view any work on the dmoz servers or network is frankly an act of Charity, hence its not likely to move at a very fast pace is it?
Thats the way i see it anyway
Meanwhile its no good editors claiming that everythings ok because visitors can still view the front end. The fact is that visitors are seeing out of date information that cant currently be changed or updated - this must be damaging / undermining the reputation of DMOZ.
Also, if volunteer editors havent been able to do any editing for over six weeks now i would have thought they would be very quickly losing interest in it or moving onto other directory sites if they love editing so much.
In all i dont think this situation is very good - Has anyone at AOL/ Netscaped passed any comment on how long before the service is likely to be up and running again?
visitors are seeing out of date information that cant currently be changed or updated - this must be damaging / undermining the reputation of DMOZ.
Do you think any visitors have actually noticed that the information is out-of-date? Answer honestly...
IMHO, webmasters usually complain that new sites they've submitted sit in unreviewed queues for months/years/decades at the ODP. Now there's an extra month or two delay, is this really the end of the world? :-)
Lets think of it this way, would the AOL systems be down for 6 weeks so no advertising could take place. I think not. So webdoctor be specific when you say that it takes time. Im not a techie, but lets say AOL affords 1-2 techs to solve the problem in DMOZ while if it was the AOL site they would have 100 techs. Its just ROI and nothing more, no need for any other type of analysis.......good faith to the browsing public just doesnt seem to be an issue. So hey as I mentioned earlier the Google directory serves up a DMOZ directory from early 2005 why not say in 2008, DMOZ itself serves up returns from 2006...No biggie.
Because they label DMoz as Google Directory..
Uhmm..no, they actually use a copy via RDF like thousands of others. Netscape/AOL hasn't taken the project serious for a long time in my opinion and should have sold it off or donated it a long time ago. The editors are the ones who keep it going - without them it would be nothing.
I think they are reluctant to start with incomplete data on the editing side and are doing work to prevent this from happening again (planned work that lost the race with the outage).
[edited by: Webwork at 1:39 pm (utc) on Nov. 29, 2006]
[edit reason] Tidying up. [/edit]
the editorial server borked big time and became unrecoverable. When the person(s) in charge of babysitting the server went looking for backups, there was none
Well there is definitely an RDF dump (http://rdf.dmoz.org/) - if we're being charitable, we can view this as a backup :-)
From what we know, when editors.dmoz.org went down, the ODP staff were in the middle of substantial work on the system. So it's not just as simple as "restore from backup, finished". We're also heard that the ODP staff have decided to complete their migration and backup work on the new system BEFORE making it available to the public. This is A Good Thing. I would humbly suggest that by and large most editors can wait - at least, that's what I'm doing.
IMHO all those people who think this kind of thing is easy should explain to us (in detail) how they would set up hardware and software to "do the ODP, but better". I would remind our readers that dmoz.org has an Alexa rank of 204. Think about the pageviews. Think about the simultaneous edits. Think about generating the RDF files from a database snapshot. Justify your answer. The best suggestion gets a bottle of bubbly from me :-)
would the AOL systems be down for 6 weeks so no advertising could take place. I think not. So webdoctor be specific when you say that it takes time.
NB: dmoz.org has no advertising and (we presume) generates no direct revenue. This should affect any budget you suggest for your proposal (see above).
Im not a techie, but lets say AOL affords 1-2 techs to solve the problem in DMOZ while if it was the AOL site they would have 100 techs.
Do you really think 100 techs will solve the problem faster? If we're talking the CIO screaming "get something back online, by whatever means", then remember that as far as the 99% of the public is concerned, dmoz.org is back online.
Its just ROI and nothing more, no need for any other type of analysis.......
If we're talking ROI then if there's no (direct) revenue, where the return on any investment?
good faith to the browsing public just doesnt seem to be an issue.
I would bet several beers that your average member of the browsing public hasn't noticed there's a problem - [dmoz.org...] is working just fine. The data is aging, but how exactly is Joe Q. Public going to notice?
[edited by: Webwork at 1:54 pm (utc) on Nov. 29, 2006]
really poor resource for Google
Some view the human involvement as a big advantage.
I've watched Google become less and less reliant on the ODP data. Today, it's all about Wiki.
Heck, even the discussions around here about ODP have continued to decline which I'm sure the ODP Editors appreciate as we're all a bit frustrated with the continued whining.
I say just move on and accept the things you cannot change. I foresee one particular directory inching closer and closer to ODP status. It will probably be another year or so, but it will most likely be the new ODP. ;)
IF AOL/Netscape doesn't really want to support the ODP then what?
Sell the darn thing. Get out now while they can make a little profit. Let someone else turn it around and make it the powerhouse that it should be.
So I think no concern for the hundreds of websites that come out between January 2006 to present that are backed in DMOZ, not to mention it takes almost years to get in...making it an absolute club. Ive heard of Meta Editors cooling their own sites, so why do we think that DMOZ is that important. Trust Rank must be low and with this latest debacle DMOZ is going to come to an untimely demise or at least will begin to wane in importance.
The only people who are upset about not getting listed are the ones chasing page rank. Face it, no searcher really goes to DMOZ and searches because it does not have up to date information.
[edited by: trinorthlighting at 3:57 pm (utc) on Nov. 29, 2006]
If DMOZ is unimportant, why do people (apparently) still get upset about not getting their sites listed?
Because many have this notion that an ODP listing is a magic bullet.
It may have been at one point in the past, but that has long since changed. A listing in the ODP is just that, a directory listing. It's no different from a Yahoo! listing or any other directory listing. And, since it is a free listing (for most), you just submit and forget it. ;)
it is because of the fact that DMOZ does not generate any revenue is why the problem is not repaired.
or the conversation could have gone like this:
AOL Tech: Sir, the DMOZ editor site is down.
AOL CEO: What?!
AOL Tech: We've got two options - a quick and dirty fix to get it back how it was, or we could use the opportunity of being offline to put a reliable, maintainable system in place.
AOL CEO: I presume the reliable, maintainable system takes a bit longer to set up.
AOL Tech: Yup. But the only people affected are our volunteer editors
AOL CEO: Who cares about the volunteer editors - we'll go for the reliable, maintainable system.