how to spot a dropped domain

[MyWifeSays]

If a .co.uk domain drops and is subsequently bought by a new owner does the "registered on" date, shown by Nominet, show the most recent purchase date?

Internic shows "creation date" for .com domains and "created on" for .org as far as I can tell. Do these reflect the newly purchased date?

I'm basically trying to spot whether domains have been dropped at some point or not and don't know if I can rely on the registration data. Any advice appreciated.

[IanTurner]

For a UK domain drops and is reregistered the registered date shown in whois will show the date of the new registration.

[MyWifeSays]

Thanks.

Does anybody know about .com and .org domains?

I'm looking at a .com one now that is for sale. It has a creation date of jun2004 but archive.org shows from jun2002 to dec2005 and then nothing since.

The domain was for sale in 2005 and the content now is completely different.

If I bought it I'd hope that Google would consider it of an age where it is not sandboxed.

Any expert views on this?

[IanTurner]

My view on .com and .org domains is that the registration date will change if the domain actually drops and is reregistered, however if it is picked up at a dropping auction which most decent dropping domains are, then it will retain the date of the original registration.

Please note I'm not 100% on this one.

[MyWifeSays]

Thanks. I guess that makes it pretty hard to spot a dropped domain from the public info.

[IanTurner]

Agree - in some tlds it is obvious whether something has dropped - .uk is an example of this, as there has to be a physical drop at the top level registration authority; but with .com, .net and .org etc then there may be a drop that is picked up at auction which never shows as a drop.

If you have a record of whois changes, together with list of dropping domains you can determine dropped domains fairly accurately.

[jmccormac]

The big problem when determining drops in Country Code TLDs (ccTLDs) such as .uk or .de etc is that they do not provide access to their zone files. The .com/net/org/biz/info etc TLDs do provide access so it is a lot easier to track the new and deleted domains. (This is part of my work - publishing domain and hoster stats.) A few years ago, most of the TLDs deleted the domains from their zonefiles and then the domains dropped. Now most of the decent domains are moved to easily identified transitional nameservers so it is still possible to track drops and auctioned domains to a certain extent. With ccTLDs, the only way to be sure is with the data from the whois. However then you get into the problem of some ccTLDs not actually including registration dates on their domains.

The Wayback Machine (archive.org) is useful to a point as some webdevs will automatically ban the archive.org spiders so a site that may have had content will not show up in archive.org. I've been tracking domains since 2000 and the number of domains in the databases here is around 220 million or so. About half of those domains are domains that have been registered and then dropped without being reregistered.

I think that Google has problems with applying its PR algorithms to ccTLDs because of the lack of zonefile access. It makes it harder to determine whether the domain has dropped and been reregistered. One case I saw a few years ago was that of a reregged ccTLD domain where the new owner kept the structure the same. The PR was unchanged - for a while anyway.

Regards...jmcc

[MyWifeSays]

Thanks for the responses. It's a very interesting subject.

Can the general public get hold of these zone files?

[jmccormac]

The gTLD registries have their own zonefile access program. Generally it involves filling out a form explaining what the data will be used for, the name and IP of the box to which the zonefiles will be downloaded. The big problem with the zonefiles is not really getting access to them. It is what to do with them when you've got access. The .com zonefile is a 1.4G download. Uncompressed, it is around 5.9G of data. The main data format is simple - just the domain name stub (webmasterworld rather than webmasterworld.com) and the nameserver. If you want to build a list of domains it is just a question of cutting the nameserver part of the line and then filtering the list so that is a list of unique domains (the uniq command on Unix/Linux does this). Then you put them into a database table and compare the list with the previous day's list to see what has fallen out of the zone. Beyond that, things get somewhat more complicated. For ccTLDs, the process is a bit more difficult because they don't give access to their zonefiles and as such you have to forensically reconstruct it . Only a few people are capable of doing something like that and they will never get a 100% accuracy on their reconstructed zonefile.

Regards...jmcc

[MyWifeSays]

Thanks jmccormac,

So even for the tld's that have zone files it's not something that can be checked retrospectively. You'd have to maintain a databse and perform a daily download.

I guess this is what those companies that sell expired domains do. Do you know if any of them offer a service to check if a domain has been dropped in the past?

Thinking about this, it's probably not something many people would use and if they did, they wouldn't pay too much. Most expensive websites have a good history which you'd expect to find in archive.org

[jmccormac]

So even for the tld's that have zone files it's not something that can be checked retrospectively. You'd have to maintain a databse and perform a daily download.

Essentially that is what you'd have to do. There are some software tools that allow you to monitor the whois data for selected domains but it is not on the same level as tracking the zonefiles.

Do you know if any of them offer a service to check if a domain has been dropped in the past?

A few. However the forum's rules forbid posting website details.

Thinking about this, it's probably not something many people would use and if they did, they wouldn't pay too much. Most expensive websites have a good history which you'd expect to find in archive.org

The complexity means that such sites are rare.

Regards...jmcc