Page Logging

trying to differentiate between hacker and crawler

[tonynoriega]

does anyone know of an application or script that can log what type of user, crawler, spider....etc is accessing a certain page?

thanks

[gregbo]

The information that is passed in the HTTP user agent can be modified by the initiator of the connection, so it cannot be trusted. In other words, something hitting a site claiming to be a particular browser, spider, etc. may be a hacker.