hutcheson - 6:36 pm on Jan 15, 2007 (gmt 0)

I've played with some ideas parallel to this, with some differences that I think would be obvious from the perspective of the volunteer editors who are the ODP community.

Clearly, what the editors get out of this is information. (And that's really all we can expect to get out of it.) In addition, the project might gain, if there's some chance of contributors eventually becoming editors.

The questions are:

(1) How to identify trusted site suggestors ("contributors")?

-- What matter whether they call themselves "professional" or not? Self-definitions are notoriously flexible. And in any case the sympathies of the volunteer editors are highly unlikely to lean towards arbitrarily reducing their fellow amateur helpers to a lower caste. (The sympathies would go altogether the other way: lean over backwards to help the amateurs; let the professionals to learn the rules on their own client's nickel or die trying.)

-- No payments: that's forbidden by the social contract.

-- What matter whether they own or fully control or merely promote the sites? Surely an honest SERP perp can be just as good a source of other people's websites as the web developer who made them? And basic human intellectual limitations pretty well guarantee that someone who's only suggesting sites they control, won't be able to suggest enough listable sites to make it worth while for us to care whether they're trusted or not.

This is really a tougher question than it looks.

--You mention "rigor", but the only rigor you mention is the 50 dollar bill, which involves nothing more than giving up a month of cable TV or a couple of restaurant visits or three DVD's.

--You mention abjuring anonymous suggestions, and that's an honorable idea. But if someone wants to suggest GOOD sites under their own name and BORDERLINE sites anonymously ... no beating around the bush, no cat-and-mouse games with the police -- I don't have a problem with that, I'd just let them do it. So, from a "value of information" point of view, I just don't think it would be worth the cost of enforcement

The only thing _I've_ ever been able to come up with is: a trusted contributor is one who's suggested some number of listed sites, and who's NOT suggested any rejected sites. (The latter is the most important criterion, but because it can so easily be spoofed by multi-aliasing, the former is also important.) When it gets down to it, trust is intrinsically not something that can generally be obtained quickly.

(2) What to do for "contributors"?

-- Distinguish their suggestions. The expectation is that editors would review such suggestions more quickly, because as more suggestions from a given contributor are reviewed, we have more information about the reliability of that contributor. But as always, the ODP doesn't have a way of forcing any kind of contributor to do any particular thing on any particular schedule, and that's too deeply embedded in the ODP culture to change without catastrophic consequences.
-- "All rejections result involving minor or correctable errors reult in ammendment demand being auto generated by scorecard being used by editor": not an issue here. Correctable errors in suggestions we already just correct--we do this for anyone except a few of the most notorious professional pests, and in any case, it's something that editors are likely to delight in doing for amateurs. Correctable issues with WEBSITES are, by definition, something we can basically never see. (How could we know whether the web developer could have added the information that we think is missing?) And in practice, such cases happen so rarely that there's no point in making a policy about them. I CAN, however, imagine an automatic re-review of a rejected site under some rare circumstances, such as a rejection that would cause a "trusted contributor" to fall into the "notorious spammer" category.

-- Feedback: I'm willing to consider rejection feedback if it benefits the directory. But I simply haven't seen a case where it would; and in this universe I haven't even been able to imagine such a case. Because basically, "rejection" carries two messages: (1) this SITE isn't worth listing, and (2) this SUGGESTOR probably doesn't understand what we're doing well enough to help -- or he doesn't care what we're doing. In either case he's been wasting editors' valuable time; and what the EDITORS most want is: no further time wasted because of so-called-help from this suggestor. And what we DON'T see is cases where a "rejection" will cause "no further time-wasting contact" but "non-rejection" will cause further contact. In other words, non-response to bad suggestions is believed (on both theoretical and practical grounds) a win-win situation for the editor and the directory.

Well, could "contributors" be bribed (with feedback) to find good sites? Maybe for each 20 points (say, 3 points for site suggested to right category, 2 points for good site suggested and accepted somewhere else)? Caveats: note that it might take time to build a reputation -- months, probably. Note that abuse might be an issue. All in all, an interesting question: to which I'm sure I don't know the answer.

There are two aspects of the ODP paradigm that are relevant here:

(1) Its center of activity is TOPICS, not WEBSITES.

(2) it focuses on privileges based on trust given to people, and not to build bureaucratic processes based on the assumptions that people can't be trusted to do the right thing but are too stupid to figure out how to do the wrong thing by the book (in the ODP context, I think both assumptions are usually wrong.)

(3) It is based on community initiative: most recent innovations in the tools have come from work that individual editors did on their own initiative -- so they could come to the community saying: "Look, here is something I know we could do (because I've been doing it) and I know helps find good sites or weed out spam (because I've found or weeded these thousands of sites). You could try it!" And then, after dozens of editors are reporting enthusiastically how well it works, it's time to begin talking about implementing more support for it into the official website.

I should add that I haven't seen a groundswell of community interest in these kind of questions, when I have raised them internally. The reality of life is that we see thousands of "potential contributors" that are building reputations as pests, and what we've historically done with "potential contributors" that built reputations as helpful folk is to just give them editing permissions. So the natural question for system-tweakers is: who is out there, that doesn't do enough work (or not enough work in one area) to be an editor, but does enough good work to build a reputation that would be useful to editors reviewing sites? And that's another question I haven't been able to answer. I don't know of any such people (which doesn't mean they don't exist, of course!)

And this proposal, less than most outside proposals, but still to some extent depends on the "ODP autokrator" imposing new "procedures" on the community. But there's really nobody who can do that; and anyone who would try to do that would lose the trust of the community immediately. So there would have to be some kind of pilot project, and I don't know how you'd do a pilot project for this. Anyone who would make a "potential contributor test subject" could just as well apply for a small local category and do live work as an editor (with access to all the basic editing tools and information--better for them and for the directory both.)