Forum Moderators: open
This impacts all the changes made since that date being used in thousands of other sites, and of course those changes will not be in (GOOGLE) <snip>
[edited by: Marcia at 10:49 pm (utc) on Oct. 29, 2002]
[edit reason] rudeness deleted [/edit]
Huh? Considering how a lot of search sites do not update nearly as often as the data dump is generated, I bet this won't be a blip on anybody's radar.
Those that need and use them frequently will obviously be concerned, those that update every three months, and want to now, will have a right to be concerned.
If ODP has a problem, and wants to say "sorry, there's been a delay" that OK - for a while .. though a decent explanation (and revised date) would be good manners.
To suggest that end users are so useless they won't even notice is just a little arrogant, but, alas, no surprise. And rather undermines the point of ODP, doesn't it?
[add]That kind of attitude gets AOL Time Warner a bad name ;) [/add]
Wrong attitude; a lot of people have come to depend on regular updates.
I've been told that generating the RDF dump is a pretty high priority for ODP. I'm sure that makes solving the problems that somhow delays generating an RDF dump a high priority as well. ;)
One month of editing isn't an insignificant change.
That kind of attitude gets AOL Time Warner a bad name
I agree with Quadrille on this. Regretably these discussions seem to end up with "us and them"
DMOZ is not all good, nor is it all bad. It has many good points, but the airing of a problem about the frequency of RDF dumps did not really merit a grouchy smack from a meta. :(
I agree however that a month's worth of data is significant.
NO site grabs the rdf and uses it right after it's generatedBut this is not the issue; many downstream sites have schedules of their own.
They may only update every three - even six - months, but the point is, they have not been able to do so, and are reduced to constantly checking when - and if - ODP is going to grace them with a delivery.
The dump may be free, but I'm not sure that entitles the giver to treat the users with such obvious contempt. If ODP cannot deliver as expected, is a prospective date (and maybe an apology) so difficult?
No, you still don't understand the process. The ODP never "delivers" -- it only "republishes" the RDF -- that is, it replaces the "last version" with the "latest version" out on the website, for anyone to download. And nobody ever "checks" or "waits" for that mythical delivery. Right now, anyone who downloads the RDF on a biennial schedule is replacing 24-month-old data with two-month-old data -- a significant improvement! And those who download on a monthly schedule (Google and AOL) are replacing one-or-two-month-old data by one-or-two-month-old data -- not such a big improvement.
But in any case the RDF is the highest priority (well, the only) public deliverable. Everything else, even the dmoz.org website, is primarily considered editors' tools (and thus of secondary importance.) So this will be fixed "soon".
How soon is soon? The ODP is a complex bit of software, and I'm constantly amazed at how well it does what it does. I have confidence in the programmers (and as a programmer, I can say that is something I DON'T have in 95% of the programmers I know).
And the standard deviation of the RDF generation period is orders of magnitude smaller than the standard deviation of the time spent in the unreviewed queue.
I and hundreds of sites are dependent upon the RDF dump and get it everytime it updates, then parse it to our sites.
A little over a month ago DMOZ had the curtesy of publishing that there was an error in the dump and that a new one would be published on a specific date.
The last dump was published on Sept 22 and THOUSANDS of people are effected by not creating the dump on schedule. GOOGLE <snip> uses the dump to update their directory, so those people will not get into it, until a dump is created.
DMOZ should examine their purpose in being...the dump should be THE HIGHEST PRIORITY.
To blow off the dump as unimportant, demeans all the work of everyone there, and the many thousands of websites dependant upon DMOZ.
To ignore the problems and not make them public displays the DMOZ MIND SET.
Let me know who to talk to, I will buy the operation...I can affort the $5.00 a month they are spending to support it.
[edited by: Marcia at 10:51 pm (utc) on Oct. 29, 2002]
[edit reason] rudeness deleted [/edit]
To blow off the dump
Just so you know, editors have no control over any aspect of the RDF dump, or for that matter over any aspect of the ODP backend. There is only one full-time staff engineer who maintains everything from the editing software to the e-mail system to the RDF generator, and she could scarcely be accused of blowing anything off. The problem is known and being worked on. I guarantee it will be fixed before Microsoft fixes all the bugs in Windows.
THE RDF DUMP IS THE HIGHEST PRIORITY. The problems are being worked on, but affect many issues. This will take time.
I guarantee it will be fixed before Microsoft fixes all the bugs in Windows.
I guess that many things will happen before Microsoft fixes all the bugs in Windows...
Pigs may learn to fly...
George Bush and Saddam Hussain may become best friends...
Google may update
I can understand why editors want to see their latest work up in glowing phosphors -- I feel that way also -- but patience in the face of what is out of my control is the only sane attitude.
Now I'm not a particularly patient person, so I console myself with the thought that the longer the update is delayed, the more edits I can get into it, so long as I just stop moping and go edit. It's like an alimentary canal -- what goes in must go out. So long as editors aren't stopped from working (as they were a couple of weeks ago) we're still pushing stuff into the pipe. When the blockage passes (as it will) all of the new improved listings will be there.
>>they have not been able to do so, and are reduced to constantly checking when - and if - ODP is going to grace them with a delivery.
No, you still don't understand the process. The ODP never "delivers" -- it only "republishes" the RDF -- that is, it replaces the "last version" with the "latest version" out on the website, for anyone to download. And nobody ever "checks" or "waits" for that mythical delivery. Right now, anyone who downloads the RDF on a biennial schedule is replacing 24-month-old data with two-month-old data ...
I fully understand; I only meant a dlivery to the collection point; apols for lack of clarity.
But that still leaves the problem; who, on this planet (other than ODP insiders) thinks that a two month old update is Good News.
Please, please, return to planet Earth and exercise a little humility - there is a problem; it is not a minor glitch inconveniencing editors, but a serious problem affecting people right down the chain. At least admit it!
(Then we can start looking for that apology ... and those flying pigs). Metas may know whats going on; most editors are as much in the dark as users ... is this our first taste of the new A-T-W Corptorate attitude?
If we email the Chairman, is it conceivable that he'll resent the smugness ... let's find out ...
... thinks that a two month old update is Good News.
Let's start with the fact that the dump is one month behind schedule, not two.
is this our first taste of the new A-T-W Corptorate attitude?
If it was they you would have seen it two years ago. As for editors, they don't work for AOL/TW, if they did I should be expecting nearly 4 years' worth of checks in the mail.
As for the users, they don't seem to be complaining (Google included). The only people that are complaining, are webmasters with a vested interest - and that's not the constituency we cater to.
Let's start with the fact that the dump is one month behind schedule, not two.
was the response to
who, on this planet (other than ODP insiders) thinks that a two month old update is Good News.
The problem with many of the threads on DMOZ is the entrenched positions of both "sides"
Forgive me if if I am wrong, but if the dump is meant to be monthly, and if it is a further month late, then as the original poster said, the update is a two month update.
The post was not claiming that the dump was two months behind schedule, it was claiming (apparently correctly) that it was a two month old update. Let's not get too aggressive with each other ...we can all learn something about the other guy's point of view
Let's not get too aggressive with each other ...we can all learn something about the other guy's point of view.
You're right cornwall.
With regards to the RDF dump, it is *supposed* to be produced on a weekly schedule. Right now it's five weeks behind.
[edited by: rafalk at 6:54 pm (utc) on Oct. 31, 2002]
The only people that are complaining, are webmasters with a vested interest
This is an interesting point. And almost certainly correct.
However...that does not in itself mean that the webmaster is wrong merely because they have a vested interest, any more than an ODP editor being a webmaster makes them incapable of reasonably objective editing.
If I might suggest, these discussions are always more fruitful if one "side" can accept that not all submitters of sites to DMOZ are spammers/crooks, and the other "side" can accept that not all DMOZ editors are corrupt.
(Though when you read the posts, you may well feel many posters believe that they must be either in the hard "they can do no right" or "they can do no wrong" camps - and that any sign of compromise is de facto a sign of weakness ;) )
However...that does not in itself mean that the webmaster is wrong merely because they have a vested interest, any more than an ODP editor being a webmaster makes them incapable of reasonably objective editing.
Of course not. The point is, however, a great deal of the posts in this thread assume that the ODP owes the webmaster some sort of accountability for producing the dump late. I merely pointed out that the *only* people DMOZ is ultimately accountable too are the end-users of the data. As long as the end-users are happy with the product (whether it is one day or one year out of date), that is the only thing DMOZ pays attention to.
The editors aren't being kept in the dark: they asked these same questions in the internal forums, and they got the information that has been passed on here. We're unhappy -- more inconvenienced by the search than by the RDF, true, but technically they're both being held up by the same problem. [But don't think the feeling is less. I PAID for those new directory changes -- with my own sweat. I want them to happen: whereas you've got 99.5% of the listings already, in the last RDF.]
Three weeks ago, editors couldn't work at all. Now THAT was a serious problem -- if the RDF had come out on schedule, there wouldn't have been anything new in it anyway. But now we can work, and 99+% of the search results are accurate. So I ask again, why the intensity of anger? And why directed at fellow sufferers?
The only people that are complaining, are webmasters with a vested interest
May be so - but the webmasters we are talking about (for a change?) are the ones who download the dump and use it on their sites.
I thought the point of ODP was to be used by those folk, and I remain amazed that they just don't seem to matter.
The 'users' aren't complaining (or are they?) because they probably do not realise why their Google etc. searches are becoming outdated ... but webmasters (including Google) do have some pride in their product. Is that really so wrong?
I know a lot of ODP editors are seriously concerned and embarassed, but not, it appears, the Metas?
Just so you know, editors have no control over any aspect of the RDF dump, or for that matter over any aspect of the ODP backend. There is only one full-time staff engineer who maintains everything from the editing software to the e-mail system to the RDF generator, and she could scarcely be accused of blowing anything off. The problem is known and being worked on. I guarantee it will be fixed before Microsoft fixes all the bugs in Windows.
For much of the past five weeks, there have been other serious problems. Some resulted from the usual sorts of things -- hardware failure, bugs in new software. Some have resulted from the growth of the ODP database and community: and so the RDF problem was NOT top priority. (While the server was down, the RDF didn't get generated, and strangely enough, nobody asked why.) Now the server is up, and the next problem on their list is -- assuredly -- about to get fixed.
<rant>Madmen running all over the place with bombs and guns, Bill Gates and all his orcs still terrorizing businesses with weekly alleged fixes for genuine security problems, Congress in session, mega-meteors hitting the planet every 65 million years or so, who knows when the Sun will go nova -- and you want me to worry about a delayed RDF dump?</rant>
> I guarantee it will be fixed before Microsoft fixes all the bugs in Windows.
So possible in the next millenia?
Part of the problem is that there are multiple programmatic steps that must be taken:
0) [stabilize hardware, etc....]
1) rebuild database
2) freeze and synchronize category moves
3) rebuild RDF
4) rebuild search
Each of these steps could take days to run in the background of our editing -- it's a large and complex database, all on one server [for other technical reasons] and we don't want to slow down the editors. Last report I heard, the "rebuild" step was failing, several days into the run. When that happens, you find the problem, fix it, and then wait several days to find the next one. It is an inherently time-consuming process. And, having worked on such multi-day program runs, I can tell you that it's more frustrating for the programmer than it could possibly be for anyone else -- days hours of sheer boredom combined with nail-biting suspense, followed by frantic minutes or hours of painstaking logical analysis.
The issue is in "good hands" (which is an expression I'd not use about 95% of the programmers and web developers I've seen).
Dear me, we are a tiny bit cynical.... looks a bit OTT.. or perhaps its a function of the crowd that you currently mix with :)
If you have "even higher standards" -- however you want to describe it -- you may still need to worry about the RDF. Other people, I hope, can relax now.
