> You should allow searchengine UAs only from the real IP addresses for those UAs.
that is what modsec is for, since you cannot keep track of the real IPs because it is not a static list.
> There's a whole bunch of IPs you should block irrespective of UA presented.
This varies from server to server. If there is a list of IPs that should be blocked on every server please post a link to it.