-- Search Engine Spider and User Agent Identification
---- June 2013 - Blocking Bad Spiders / Bots / User Agents
lucy24 - 12:59 am on Jun 24, 2013 (gmt 0)
also opens the door for bots to change their procedures as they become aware of the methods.
I'm not concerned with bots appearing to be human. What I'm very concerned with is humans misidentified as robots. Hence the reference to "false positives" vs. "false negatives".
It's a subject particularly close to my heart because my own browser for obscure technical reasons insists on identifying itself as "like Firefox 3.6" which is enough to fall afoul of some sites' user-agent detection. Generally not in access but in the features the site makes available. (Google's image search-- which I never use-- gives me the old format although the browser is perfectly capable of handling the scripting. TV guide has stopped working entirely; I've had to shift the bookmark to Safari.)