Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: phranque
that don't involve maintaining a list.There's always a list. It might be a list of IP addresses, or it might be a list of User Agents, or it might be a list of header fields and values, or it might be a list of behaviors. Any time you're matching something against something else, there's ultimately a list involved.
use the scraped site for your own branding.
Your company is blocking Merck?Have you ever had a visit from Merck? Or Halliburton or DuPont or Eli Lilly or any of the other companies that started out controlling a whole /8 before discovering they hardly need any of it--and certainly not externally--so it's more profitable to sell to AWS?