homepage Welcome to WebmasterWorld Guest from 54.237.71.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Using Project HoneyPot and Mod Security
Blocking spammers, scrappers, and hackers.
frontpage




msg:4533980
 3:11 pm on Jan 7, 2013 (gmt 0)

Using Project HoneyPot and Mod Security

I am experimenting with a new facet of ModSecurity v2.7.x, the ability to use Project HoneyPot's HTTP Blacklist (HTTPBL) of bad IP addresses to reduce attacks on our server.

What is Project HoneyPot?


Project Honey Pot is the first and only distributed system for identifying spammers and the spambots they use to scrape addresses from your website.


How, What, When, Where?

1) In order to do this, you need to have ModSecurity 2.7.x installed on your server. MS 2.7 has a new variable called "SecHttpBlKey" which uses your Project HoneyPot Access Key.

2) You need a Project HoneyPot access key. It's free.

3) You need to add the following ModSecurity Rule to your rule list for it to work.

Example:

SecHttpBlKey INSERT YOUR ACCESS KEY HERE
SecRule TX:REAL_IP|REMOTE_ADDR "@rbl dnsbl.httpbl.org" "id:'99010',chain,phase:1,t:none,capture,block,msg:'HTTPBL Match of Client IP.',logdata:'%{tx.httpbl_msg}',setvar:tx.httpbl_msg=%{tx.0}"
SecRule TX:0 "threat score (\d+)" "chain,capture"
SecRule TX:1 "@gt 20"


4) Restart Apache

I am not getting plenty of log hits that look like this now.

Access denied with code 406 (phase 1). Operator GT matched 20 at TX:1. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "343"] [id "99010"] [msg "HTTPBL Match of Client IP."] [data "RBL lookup of MYACCESSKEY.82.225.47.96.dnsbl.httpbl.org succeeded at REMOTE_ADDR. Suspicious comment spammer IP: 1 days since last activity, threat score 82"]

I like that it blocked this IP which is labeled as a "Dictionary Attacker" in the database. "The Project Honey Pot system has detected behavior from the IP address consistent with that of a mail server and dictionary attacker."

Access denied with code 406 (phase 1). Operator GT matched 20 at TX:1. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "343"] [id "99010"] [msg "HTTPBL Match of Client IP."] [data "RBL lookup of MYACCESSKEY.99.226.10.5.dnsbl.httpbl.org succeeded at REMOTE_ADDR. Suspicious IP: 64 days since last activity, threat score 25"]

What is it doing?

Basically, any IP that is on the HTTP Blacklist (HTTPBL) will be served a 406 Server Response code.

So far it has not put that much strain on the server as far as I can tell and hopefully it will reduce scrappers.

I am still looking for more documentation on this new rule.

 

wilderness




msg:4533993
 3:35 pm on Jan 7, 2013 (gmt 0)

Harvesters (bots or otherwise) that visit websites and email spammers are two entirely different creatures.

An effective method of gauging this difference is to start a blog run with PHP and track those email spammers, then compare those spammers two your regular websites harvesters.

It's two different worlds.

frontpage




msg:4534023
 4:52 pm on Jan 7, 2013 (gmt 0)

Sigh....

For those who actually read what the HTTP Blacklist (HTTPBL) does, it contains the following banned IP's.

Http:BL is similar to a DNSBL but for web traffic rather than mail traffic. This data can be used in order to stop malicious robots from accessing your web pages.


harvesters
bad web hosting companies
malicious robots
search engines
dictionary attackers
comment spammers
spam servers

It's just not for email spammers.

Example: "Rule Breakers" robots in the black list.

Kavande Crawler 1.0/Nutch-1.4 ( Iranian National Web Crawler)
parsijoo
parsijoo-crawler

wilderness




msg:4534033
 5:31 pm on Jan 7, 2013 (gmt 0)

For those who actually read what the HTTP Blacklist


There's nothing worth reading there.

Glad it's effective for your needs.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved