GoogleImageProxy

redux; runs amok

[Pfui]

- Last discussed almost exactly a year ago: [webmasterworld.com...]

- Same UA as usual, but behavior definitely not usual:
Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0 (via ggpht.com GoogleImageProxy)

- Last night that thing suddenly ran amok in concert from three Google IPs, hitting a total of 1,444 times in 40 minutes to favicon.ico and redirect files Google's never been allowed to access in the first place. (So much for robots.txt...)

To prevent further abuse, we kill-filed the offending IPs, for now:

66.249.82.182 (a.k.a. google-proxy-66-249-82-182.google.com)
66.249.82.184
66.249.82.186

Bon voyage into the ether, bots behaving badly.

[tangor]

I'm also seeing a rise of bad actors via google-proxy, all apparently based in Brazil... been mulling over banning all google IPS EXCEPT the bots I want.

[tangor]

Note: Bing is not that much better, either!

[dstiles]

In general I block browser versions before a certain number, usually about ten before the latest decade. If people are not up to date, that's dangerous. If bots are not up to date, their problem.

[Pfui]

tangor: I've whitelisted only Googlebot UAs from googlebot.com for ages. But it's tricky because the same address can spew different UAs in the same session. Have not noticed a particular problem with Brazil. Yet.

Thinking out loud... Agreed re Bing and basically most MS IPs. Bing still tries to hit disallowed files -- after 20-plus years! But every time I think about axing them, I realize others' links to/mentions of us will appear up top, including scraped copy -- for the handful of people who use Bing. I've not seen a single real visitor referring from a Bing search this year. Yet Bing still hits scores and scores of times a day.

dstiles: Yep. Every year or so I review how far along I block major UAs by age -- through version 50? 55? Also, am amazed how many non-major developers just piggyback onto strings dated 2010. Come on!

All: FWIW, no massive assault from GoogleImageProxy last night, whew.

Just the usual 100 or so google-proxy- hits from 27 different IPs. (And another 50 or so crawl-66-249- hits from googlebot.com. Plus one Googlebot faker from fully-denied googleusercontent.com.)

[dstiles]

I did notice, after posting above, that Waterfox is still using Firefox/58 in its UA. How did I find out? I tried to load one of my extra-restricted sites and wondered why it failed. :)

[lucy24]

browser versions before a certain number

I don't go purely by number--unless it's something plainly ridiculous, like single-digit Firefox or MSIE 6--but I periodically check whether I have seen any humans using a given range. The old browsers used by humans with old computers aren't necessarily the same browser version numbers claimed by unwelcome robots. And I’m certainly not going to bar people just because their computers are old: so’s mine, after all. (I live in dread of the day it dies and I'm no longer able to use all the outdated software that I like better than the newer versions.)

Waterfox is still using Firefox/58 in its UA

Well, FF/58 is not awfully old. But I did recently block
Waterfox/56.0.4
after seeing that nobody but humanoid robots was using it. (“Humanoid” in this context means not only a human-seeming UA, but the full complement of humanoid headers.)

[dstiles]

My waterfox is Waterfox/56.2.12 but the latest version is /56.2.13 so close enough. Alpha version, for some reason, skips to /68.0 but no idea what Firefox version it will be.

[Pfui]

Lucy said: (I live in dread of the day it dies and I'm no longer able to use all the outdated software that I like better than the newer versions.)

Ditto! So I watched eBay for ages till I found the exact same machine from a reputable seller for a reasonable price -- it took a while -- and bought it. Now I've got an identical set-up if/when the time comes and the machine conks out. Over the years suddenly dead motherboards and such have been both pricey and time-wasting, Now, all I do is plug in the external back-up drive and I'm good to go.

(Then again I may be too into preventive redundancy... I'm sitting next to a 10 y.o. MacBook Pro, a 7 y.o. iMac, and a 1 y.o. MacBook Air, each running its maximum OS. But the thought of losing older functionality, and program-specific files, gives me the willies.)

/tangent :)