I read some good posts, in particular the sticky: Quick primer on identifying bot activity
My "useless" traffic has increased ten fold and needs to be reigned in... and I am wondering how to get rid of the bandwidth thieves.
I still use ASP and was wondering on how to logically / structurally tackle this problem. E.g where do I let any filters /loggers do their work? What to log?
On an IIS server I have global.asa, then 404 and 500 handling.
I saw some silly form posting attempts and just implemented encrypted time stamps, and a CSS-hidden field. There is separate logging of 500 and 404 events. The "logging" becomes a mess, being in different places, and at some stage I can't tell when what code kicks in. (programming is not my profession)
Is there a best practice design or approach to catch the non-sense?
1. check incoming IP against blocked IPs, if on deny
2. check if URL contained characters not used on website; e.g. percent sign in URL trigger "put on IP block list"
3. if form is posted with expired token > onto block list
4. if form has honey pot filled in > block list
Reading that UAs change or can be anything, is there value of checking them? Why? What for?
FORWARD-X being captured, and reverse DNS look-ups being done, when? What triggers them? and what info to store about it?
Are these sensible questions to ask, or would the answer give too much away?
The more I read, the less I know, and the more daunting the task :(
And there are Anti-hot-linking mechanisms, etc. needing to be included as well?!
Any hint or pointers to literature appreciated, on how to build successful anti-bot anti-nonsense solutions.
Or what method in which order and at which location need to be employed to build such a solution?