Welcome to WebmasterWorld Guest from 3.80.60.248

  1. Home
  2. Forums Index
  3. Search Engines / Search Engine Spider and User Agent Identification

Forum Moderators: Ocean10000

Message Too Old, No Replies

another plainclothes bingbot?

     

lucy24

7:47 pm on Sep 15, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15444
votes: 739


Showed up abruptly on 12 September: 

40.77.169.100 - - [12/Sep/2016:17:12:27 -0700] "GET /ebooks/bourquin/ HTTP/1.1" 200 8391 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 
40.77.169.101 - - [12/Sep/2016:19:47:31 -0700] "GET /ebooks/kleinschmidt HTTP/1.1" 301 564 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 
40.77.169.101 - - [12/Sep/2016:19:47:31 -0700] "GET /ebooks/kleinschmidt/ HTTP/1.1" 200 5297 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"

Identical pattern--even the same times, give or take ten minutes--on the 13th and 14th.

I've got a long-standing lockout on the plainclothes bingbot, but (a) it doesn't cover the 40.77 range, and (b) it's for the MSIE user-agent.

A possibly-bing-related entity calling itself Chrome is almost as funny as the mobile bingbot calling itself iPhone.

Edit: Eeeuw. I'm not used to viewing the Forums in Firefox. What language does the color-coder think the "code" is in?

not2easy

8:22 pm on Sept 15, 2016 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4159
votes: 262


I see the identical UA as a humanoid visit from a Verizon IP this month. I have not completed all logs yet. From a nearby but different 40.77.167.80 I see a Bingbot UA blocked itself by ignoring robots.txt (after requesting it 3 times in a row.)

I don't have anything (yet) with that UA in that range, but as mentioned, I'm not all up to date yet.

keyplyr

8:25 pm on Sept 15, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


While the range is registered to Microsoft Corp, it is not a designated crawl range so the visits to your web site could really be unrelated to Bingbot. I assume "plainclothes bingbot" is just a moniker you give to anything unexplained from M$. As you've noted, M$ has always done this stuff.

lucy24

4:17 am on Sept 16, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15444
votes: 739


I assume "plainclothes bingbot" is just a moniker you give to anything unexplained from M$.

Come to think of it, these visits have shown pure robotic behavior: html alone, nothing else. Other bingoid entities (such as the notorious Drake Holdings) also get styles and scripts, though not images.

Simon_H

11:58 am on Sept 28, 2016 (gmt 0)

Full Member

Top Contributors Of The Month

joined:Sept 28, 2015
posts: 273
votes: 171


Was about to start a thread on this and came across this one!

Seeing something similar - the IPs are Bingbot, but the UA is wrong, so they're messing up the custom tracking we have as our tracking uses the UA to determine bot exclusions. They include IP ranges 131.253.xxx.xxx and UA is Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0; Trident/5.0).

They're hitting lots of legacy product pages on the site, so this feels like normal bot activity, just with the wrong UA. Or maybe I'm missing something very obvious?
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members

  1. Home
  2. Forums Index
  3. Search Engines / Search Engine Spider and User Agent Identification