They're using a different bot than previously seen: [webmasterworld.com...]
lucy24
8:37 pm on Aug 29, 2018 (gmt 0)
Earlier this year I tossed Qwantify back from Authorized to Re-Evaluate, probably because I caught them requesting pages in a roboted-out directory. They may turn out to be one of those annoying robots that appear to honor robots.txt when you deny them by name, but will otherwise ignore its directives. (This may well lbe attributable to stupidity rather than malice.)
Looking them up now, I noticed a detail: 194.187.170.abc: Mozilla/5.0 (compatible; Qwantify/2.4w; +https://www.qwant.com/)/2.4w 194.187.171.abc Qwantify/1.0
In the previous thread I'd noted the alternative UA, but I don't think I noticed that the two came from different IPs within the same range. The shorter form gets the favicon or, very rarely, images; the longer form gets pages. There are far too many for this to be coincidence. (Linguists call it Complementary Distribution. I don't know if it's got a technical term of its own in the present context.)
I also find a handful of requests from entirely different neighborhoods (one in 54, others in 217.182.90) with the UA Mozilla/5.0 (compatible; Qwantify/vR; +https://www.qwant.com/) Is this what happens when you go on github?
Is your newly noted bot coming from all over the /22, or from a smaller subsector?
keyplyr
9:27 pm on Aug 29, 2018 (gmt 0)
Is your newly noted bot coming from all over the /22, or from a smaller subsector?
