Stumbleupon?

[aristotle]

I noticed this odd-looking entry in Latest Visitors for one of my sites:

Host: 199.30.80.103
/
Http Code: 200 Date: Jun 04 22:51:12 Http Version: HTTP/1.1 Size in Bytes: 20765
Referer: -
Agent: Mozilla/5.0

/image.gif
Http Code: 200 Date: Jun 04 22:51:13 Http Version: HTTP/1.0 Size in Bytes: 1074
Referer: -
Agent: Python-urllib/1.17

Here is the IP lookup info:

IP: 199.30.80.103
ISP: StumbleUpon
Organization: StumbleUpon
Services: None detected
Type: Corporate
Assignment: Static IP
State/Region: California
City: Santa Clara

I see Stumbleupon in my logs pretty often, but it normally identifies itself, so not sure about this case.

Question: My impression is that Stumbleupon shows your page in a frame on their site. Is that the reason why traffic from them isn't worth much? Or is it just that it's generally poorly targeted?

[lucy24]

Services: None detected

Haha. I see StumbleUpon once in a blue moon for my game-related pages. It's got something to do with human recommendations, so I've been inclined to let them come and go as they please.

[aristotle]

Lucy -- I posted this because it doesn't look like a normal Stumbleupon referral. I think it might just be a bot checking a page in their database, maybe to see if the page has changed, but if so, wondered why it didn't identify itself.

[lucy24]

What struck me looking more closely is that it's got two different user-agents, one humanoid and one robotic, and two different HTTP versions (1.0 and 1.1). You'd think switching protocols would be more trouble than it's worth.

Is it literally just "Mozilla/5.0" or was it a full humanoid string beginning with Mozilla? I block "Python" in the UA string, so that one wouldn't have gotten in. Also ^Mozilla$ (complete) but so far haven't had cause to block ^Mozilla/some-number-here$. I know there's a "Mozilla/4.0" that's some kind of archiver.

:: detour to check old logs for "Mozilla/[\d.]+" (literal quotation marks to bring up full UA string) ::

Wow, no idea there had been so many. Most slipped under the radar because they were 403d from the get-go for various reasons such as blocked IP, or POST request not involving the contact page. Thanks to the form of my search, I also find some JikeSpiders from years ago:

"jikespider \"Mozilla/5.0"

(literal text from logs, which explains why the string \"Mozilla is blocked. I'd forgotten the reason.)

Does StumpleUpon answer email? On rare occasions over the years I've sent off a "please identify your robot" letter to assorted places, and on still rarer occasions I get a reply.

[aristotle]

Thanks for going to so much trouble Lucy. I didn't expect you to do that.

Anyway, I can't answer most of your questions, except to say that those are the exact UAs as they appeared in Latest Visitors, which normally just duplicates what's in the raw logs. Also, I think I remember occasionally seeing a Stumbleupon bot which did identify itself, but am not 100% sure.

[lucy24]

Well, I was curious too.

74.201.117.226 - - [05/Sep/2011:09:54:16 -0700] "GET /games/index.html HTTP/1.1" 200 4510 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0.1; StumbleUpon; [e-mail]) Gecko/20100101 Firefox/4.0.1"

Next visit was from 38.99.102.237, but by mid-2012 they'd settled into 199.30.80.abc (range looks like 96-127 i.e. /27, though notes say they control the whole /22), which is where we came in. And, ahem, by mid-2012 I'd instituted an index redirect ;) Last seen, only a week or two back, by which time they'd changed to a marginally more plausible

Mozilla/5.0 (Windows NT 6.1; rv:12.0; StumbleUpon; [e-mail]) Gecko/20100101 Firefox/12.0

[aristotle]

For comparison, here's a normal Stumbleupon referal that came in today to the same page:

Host: 76.84.253.8
 / 
 Http Code: 200 Date: Jun 06 16:36:44 Http Version: HTTP/1.1 Size in Bytes: 20765 
 Referer: http://www.stumbleupon.com/su/8ZVgst/www.example.com
 Agent: Mozilla/5.0 (Windows NT 6.0; rv:38.0) Gecko/20100101 Firefox/38.0 
 
/image.gif 
 Http Code: 200 Date: Jun 06 16:36:45 Http Version: HTTP/1.1 Size in Bytes: 1074 
 Referer: http://www.example.com 
 Agent: Mozilla/5.0 (Windows NT 6.0; rv:38.0) Gecko/20100101 Firefox/38.0

[