-- Search Engine Spider and User Agent Identification
---- 150.70. revisited
lucy24 - 8:05 am on Sep 18, 2011 (gmt 0)
I only remember them fetching main files like .htm, .html, .zip, and not .css, .js, but if they are fetching those others now, it would be a reasonable escalation because malware is being stored in those nowadays, too.
I'm surprised they're not picking up the jpgs. Apparently you can stash anything in those. Had one recently that was fatally damaged, but the desktop preview was perfect-- because the preview data was intact. (This was, of course, a complete jpg, not a "web-ready" one.)
Like any AV, it inspects the file after the user's browser has received it and written it to hard disk in the browser cache. That's normal AV activity.
Well, "normal" is a statistical word. If you've already examined the cached copy, what else do you need? Heck, their own user could have planted malware on me in the course of their visit. And then the cached copy would be clean and the hours-later version would be dirty-- but I'm not their paying customer.
:: vague train of thought involving Public Health notifications ::
If you try to go to one, it blocks your request even before you can get a Google/Firefox Safe Browsing message or Internet Explorer warning.
That would be like, uhm, the caution my browser used to put up every time I logged on to Bing Webmaster Tools? (They must have changed something, because the browser has stopped worrying.)
Come to think of it, I might be able to identify the specific humans involved in a couple of these visits. Would be interesting to know how it works from the user's side. Though I doubt they'd be able to explain the lack of a transparent UA name like "AntiVirusBot" so you know what you're dealing with.