jmccormac - 6:50 pm on Nov 14, 2002 (gmt 0)

bird posted:

The assumption that started this thread seems to be that a check once a month would be the bare minimum. Now assume that the ODP ran a spider to hit the whois system 3 million times a month. How fast do you think netsol would block that spider?

I think you are getting DNS/nameservers mixed up with WHOIS results bird. Checking the nameserver details and the webserver IP details of a site when it is submitted is the easy part. Any deviation could be detected easily. The rough .com count from Dmoz RDF is 1101549 and this is not that difficult to check. It would take a few hours to do it against the zonefiles.

Yes, that's obviously the easy part of the problem. The hard part is to compile a complete list of all squatted domains every month. There's no good and efficient way to do this unless you're a domain registrar yourself. And even this ignores the fact that the ODP has domains from pretty much all country code TLDs in its database, each of which may require accessing a different whois server with a different syntax.

You don't have to be a domain registrar to compile the list of squatted domains. There are easier ways but a knowledge of DNS operations and domain issues is essential. Otherwise it is all back to individual surfers detecting squatted sites and notifying Dmoz. The immediate problem is cybersquatting in the CNO tlds. These can be tackled first since they are pan-national and the easiest targets for cybersquatters.

Many people really massively underestimate the scale of the technical challenges that an operation like the ODP poses to one lonely technical staff person.

I don't underestimate the the challenges of finding domains and linkswamps nor do I underestimate what is involved with Dmoz. (I had to implement a static version earlier this year for an Irish website.) Part of the work I do is identifying which .com/.net/.org domains are registered by which country. The domain problem with Dmoz is not that complex and since you can establish a start point, there is a solution. It may not be 100% and that is where the users would come in. I don't dispute the good work that has been done but to think about domain/DNS issues does require a somewhat different approach to the one that has been used to date.

Regards...jmcc