Crawl whois

how would one do that then

[Crush]

Hi Wise ones

How would one crawl whois in its' entirety without them banning your IP etc.

[Lord Majestic]

For what purpose?

[Crush]

extract domains, expiry dates of site I could be interested in buying

[httpwebwitch]

Don't crawl whois.
If you want to know if something is available, you check. Crawling whois will tell you what is, not what isn't, registered.

Why not get one of those lists of expired domains - ones that were once owned but were allowed to lapse. those are all over the place, and they're free.

If you need a bulk service to check domains, then you're probably an ad spammer. You could hire a million monkeys to do it for you.

[Crush]

No hypothesizing about what I may or may not be doing ;)

So lets say hypothetically possible or not.

I would say crawl with proxies but then if you get stopped after 10 goes it would take for ever to get what you want plus you would need one mother list of proxies.

[Brett_Tabke]

Talk to someone willing to share their zone files...