tangor - 10:55 pm on Dec 16, 2011 (gmt 0)
That 500 error might be an upload error... needs to be ascii not binary!
You can reduce your IP entries significantly if you are blocking the "pagename.php" request with 403 and ignore the 404s (filter both out when doing stats), but only if those ips are NOT ALSO hitting other pages on your site in a bot-like manner.
For popular vulnerability test requests I 403 those and move on, then weekly run a "403" report of those ips and match them to the rest of the logfile... makes it easier to spot some bad actors, THEN I might issue ip deny from to block a range or range sub-set. The rest of the time I use UA and sometimes REFERER (sic) to fine tune. A much smaller .htacess file is the result (about 200 lines, including blanks).
On one site I don't use .php for anything so blocking that--alone--reduced the bad actors over 50%. Study your logs (excel's pivot report function is a handy tool!) and gather more data.