mihomes - 8:18 am on Oct 22, 2012 (gmt 0)

So I get the following in an email :

Time: Sun Oct 21 10:31:52 2012 -0400
IP: 66.249.73.70 (US/United States/crawl-66-249-73-70.googlebot.com)
Failures: 5 (mod_security)
Interval: 300 seconds
Blocked: Permanent Block

Log entries:

[Sun Oct 21 10:28:28 2012] [error] [client 66.249.73.70] ModSecurity: Access denied with code 501 (phase 2). Match of "rx ^((?:(?:POS|GE)T|OPTIONS|HEAD))$" against "REQUEST_METHOD" required. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "38"] [id "960032"] [msg "Method is not allowed by policy"] [severity "CRITICAL"] [tag "POLICY/METHOD_NOT_ALLOWED"] [hostname "example.com"] [uri "/"] [unique_id "UIQGjGB-guIAAAwRPyQAAAAB"]

Now, I have CSF installed so after this happens a few times it does a perm ip block. So, it perm blocks googlebot until I manually remove that block.

I looked into the error log of apache and it is always something like so :

[Fri Oct 19 03:34:57 2012] [error] [client 66.249.73.70] ModSecurity: Access denied with code 501 (phase 2). Match of "rx ^((?:(?:POS|GE)T|OPTIONS|HEAD))$" against "REQUEST_METHOD" required. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "38"] [id "960032"] [msg "Method is not allowed by policy"] [severity "CRITICAL"] [tag "POLICY/METHOD_NOT_ALLOWED"] [hostname "example.com"] [uri "/"] [unique_id "UIECoWB-guIAAHJ5BZoAAAAD"]

[Fri Oct 19 03:34:57 2012] [error] [client 66.249.73.70] File does not exist: /usr/local/apache/htdocs/501.shtml

I did some testing and have come to the conclusion that this is caused by crawling https locations. I have an ssl installed on a site, however, I no longer use it. All files have been removed other than the htaccess in its root dir.

Some more testing and I found that if I try to view https of any of my sites the mod_security kicks in and I block myself. Also, when I try to view any https location of my sites (any site) the browser gives back a message how it cannot connect or the there was a failure.

So, I have two questions :

1 - Shouldn't this be throwing a regular error page like a 501 instead of this connection error stuff (I have error pages setup for all cases). I actually made a 501.shtml file in /usr/local/apache/htdocs/, and while that error does not show any longer it still does not serve the error page. Let alone, shouldn't a default kick in if its not present... in that folder I have 400,401,403,404,500, and now the newly created 501.

2 - How can I stop mod security from doing this for https. I was able to block myself and I certainly do not want G-bot blocked.