mag-rob03.osl.basefarm.net mag-rob14.osl.basefarm.net etc.
dozens of them eating up my bandwidth. Anyone know the useragent name for robots.txt?
Pfui
8:46 pm on Apr 20, 2010 (gmt 0)
I've blocked basefarm.net (parent: basefarm.no; a server farm), for as long as I can remember.
Seeing as how the company's, or one or more of their account-holders', bot-running is repeatedly abusing your resources, using cloaked user-agents, and ignoring common robots.txt rules, whatever they're using probably wouldn't heed agent-specific rules anyway.
