gregbo - 2:42 am on Jul 3, 2006 (gmt 0)

I whitelist all access to my server based on user agents and MSIE, FireFox and Opera get thru as well as 5 major search engines and EVERYTHING else is bounced.

I imagine this policy would reduce some click fraud but it doesn't work in the general case, and you could very well be turning away legit, even potentially converting traffic.

No, I don't site around blocking things, they simply never get to my content in the first place unless I let them. I get a report of anything new trying to access my site and review who/what they are to see if I should let them in next time.

This doesn't scale for a heavily trafficked site.

Which is also why my web site checks all inbound IP's for an open proxy as well as blocking the known list, and I block accesses that come from many server hosting farms as well for the very fact people use cheap hosting to put various automated tasks and proxies online.

IP addresses change hands. Companies are bought and sold; infrastructures are merged. What might be registered to a server farm one day might be registered to a block of broadband users the next day.

Basically, if it's coming from an IP address that humans don't use, like hosting farms, or where it's vulnerable like proxies, I block it.

How does one determine what is an IP address that humans don't use? (I'm restricting this to the set of addresses that are on publicly routable networks, not the private or link-local address spaces.)