Forum Moderators: open

Message Too Old, No Replies

Getting hammered by Bingbot

         

tangor

4:25 pm on May 17, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Since May 2nd I have been getting hammered (over 68000 hits) for filenames NOT on my server. They are in the form of

/xxxxxxxxxx where x is any number 0-9, always 10 digits.

More strange, while majority are 404, a significant number are showing as 301!

What is going on?

Meanwhile, I want to ensure that any 301 (other files are affected as well) is never processed as I HAVE NO REDIRECTS IN FORCE or is returned as a 404-410. Point me in the direction to learn enough regex to make any request for /xxxxxxxxxx (any integer) die a horrible 404 death. What I know would fill a gnat's thimble, and that's a lot smaller than a regular thimble.

Kicked into overdrive last night in the 301s began to take in existing 200 files on the system!

All IP addresses checked so far resolve to MS, though I did locate at least one that was G.

.htaccess is clean ... has nothing I did not put in it.

Site moved from host where it lived for 15 years April 13 with no problems. Logs were normal until recently.

Asking here since the villain involved is:

Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

What is most perplexing is the site getting hammered only has 762 pages, including css!

jmccormac

4:40 pm on May 17, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Bingbot ranges or MSFT's Azure cloud? Microsoft tends to have some rather dimwitted and completely amateurish approach to dealing with sitemaps and will hammer away at site. The /xxxxx thing is a bit unusual. Did you do a search on Google or Bing to see if there are such links from other sites? Also, do these IPs have reverse-dns set up so that they resolve to .search.msn.com ranges?

Regards...jmcc

tangor

5:28 pm on May 17, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Combination of both (bing and Azure) ... as for the ips tested, just pasted into reverse lookups and all resolve to search-msm or at least one azure. Checking ips past xxx.xxx might reveal other things?

What is crazy for me is that up until last night these /xxxxxxxxxx requests were getting 404s and NOW have changed to 301!

lucy24

5:52 pm on May 17, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



a significant number are showing as 301
Nothing unexpected there: It's asking for the wrong www and/or wrong https; the server doesn’t go looking for the (nonexistent) file until the request is otherwise acceptable.

Pattern:
^\d{10}$

tangor

6:06 pm on May 17, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That might be an answer ... this site lay fallow for a zillion years ... had to move it. In process of that did some updates. Just recently installed:

#Force non-www:
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www\.example\.com [NC]
RewriteRule ^(.*)$ http://example.com/$1 [L,R=301]

I suppose I should just wait this out?

As for the regex on the pestiferous /xxxxxxxxxx, THANKS!

Montresor

6:21 am on May 18, 2019 (gmt 0)

10+ Year Member



Getting hammered by Bingbot


Microsoft should rebrand it to Bangbot?

tangor

12:50 pm on May 18, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@Montresor ... that deserves a trip to the woodshed for ... cake and ice cream! :)