Microsoft Bot Processing jQuery?

[TheMadScientist]

I've seen some Microsoft posts in here, but...

I have a jQuery based JS browser sniff script that got fired this am on multiple 'unrelated' pages of a site (EG state+zip-code for multiple 'not close together states and zip codes') and I didn't think much of it for the first couple, but then I saw the pages they were accessing made NO SENSE in the order they were requested the referrer was always blank and the same browser hash appear from 3 different IP Addresses ... The hash is an sha1() of the browser info sent to a php script using jQuery, so I went and did a look up...

The 3 IP Addresses were:

65.52.104.NNN
157.55.18.NNN
207.46.193.NNN

2 are Microsoft Corporate; 1 is msnbot-IPADDR-search.msn.com

The browser configuration is: Mozilla 4.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.40607; .NET CLR 3.0.30729)

Screen Height: 768px
Screen Width: 1024px
Color Depth: 32 Bit
Etc.

Is someone spoofing, or is M$ really processing jQuery like it seems?

You have to be able to process jQuery and make AJAX requests for me to get the info I have ... There's no other way it ends up in the DB.

[dstiles]

Could be looking at content using a "browser bot"? Something that behaves like a browser but is really a bot?

I have 8 IPs self-blocked many times in the 157.55.18.NNN range (42 to 50), apparently for using a bad header. Likewise 207.46.193.NNN (42 to 53) but for a slightly different reason. Probably both were masquerading as browsers. There are doubtless a few other IPs self-blocked.

I have 65.52.104.NNN listed as mostly bot rDNS and have no easily-read record of problems on those IPs, but if it's not a bingbot or msnbot UA I would silently reject it on those IPs. Stuff happens. :)

[TheMadScientist]

Yeah, I watched for a bit longer and looked a bit closer ... 14 different IP Addresses in under two hours all looking like they resolve to Microsoft blocks (I didn't run them all, but the ones I did do) pretty much says they're running a 'browser bot' and it actually runs jQuery ... There's a variable I have in the script that's pulled from the source code of visited pages using $('#id').html() and it's set correctly for the entries, but I doubt a person / group of people bothered with 14 different IP Addresses for 20+ unique page requests in a relatively short time period.

[edited by: TheMadScientist at 10:04 pm (utc) on May 1, 2011]

[Leosghost]

The thot plickens..I haven't seen any evidence ..but then I wasn't looking for it ( will do so from now on ..nice "heads up" TMS :)..but it would be useful for some things I'd like to do if they and other SE could and did process jQuery.

[TheMadScientist]

I wasn't really looking for it either, one of those times right after I installed something and wanted to have a look and see how things were going ... A bit more info is: no cookies; no referrers; no link clicks ... It would be nice on some of the stuff I do too, so I'm not even going to block it; way more interesting to see how 'it' behaves.

BTW: It's still requesting pages from the site 9 hrs after I first noticed it ... It's definitely a bot, imo.

[TheMadScientist]

New UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 1.1.4325; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30707; InfoPath.2)

Same deal: M$ IP Address.