joined:Mar 9, 2013
Recently I implemented a bad bot blackhole similar to the one listed on perishablepress. (http://perishablepress.com/blackhole-bad-bots/)
A quickie bg on my site: Been negative seo spammed and had my content stolen and pasted on thousands of sites. Over 1 million backlinks all spam. The latest google update restored my site from what I believed to be its final resting place in the sandbox.
Then, like clockwork, the spam started all over again. So, I decided to implement a bad bot blackhole- which I finished implementing with cloudflare and some special tweaks three days ago.
To date, I have blocked over 340 ip addresses. I was initially concerned I was blocking legitimate visitors. But after checking about 30 ips and them all turning up listed in various spam lists I decided it was working properly.
Some interesting things I found were one, nearly every user agent that has been blocked is this:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0
Seriously - almost every one. Do you guys have any idea why that would be? Secondly - I would like a way to bulk loopup all these IPs that have violated my robots.txt - just to see what percentage of them are blacklisted.
Anyone have any tools or sites that can do this for free? Looking them up one by one is painfully slow and inefficient.
I highly recommend everyone implement some form of the bad bot blackhole. Especially if you have spam issues.