Blocking Weird Traffic

[ichthyous]

---

System: The following 3 messages were cut out of thread at: https://www.webmasterworld.com/google/4957935.htm [webmasterworld.com] by goodroi - 10:28 am on Aug 7, 2019 (utc -5)

---

Several of you have reported a spike in traffic from Chicago... That is relativity.com. I blocked their IP address but I'm still getting hit by Chicago traffic with a 99% bounce rate all day. It's coming from unknown.unknown. Anybody know how to block this... It doesn't seem to be coming from any one IP, I checked my server stats

[ichthyous]

@Treud the relativity IP I blocked is 23.101.169. 3

[Kshamaster]

May be it's a DDOS attack

[oleotio]

@Treud the relativity IP I blocked is 23.101.169. 3
[abuseipdb.com...]

[edited by: goodroi at 3:33 pm (utc) on Aug 7, 2019]
[edit reason] thread formatting [/edit]

[NickMNS]

@Treud and Oleotio

IP I blocked is 23.101.169. X

Take a look at this thread: [webmasterworld.com...]

From that thread, there is some indication that this is or may be Bingbot.

[edited by: goodroi at 3:33 pm (utc) on Aug 7, 2019]
[edit reason] thread formatting [/edit]

[oleotio]

@Treud and Oleotio

IP I blocked is 23.101.169. X


Take a look at this thread: [webmasterworld.com...]

From that thread, there is some indication that this is or may be Bingbot.

Maybe it's some Microsoft service someone's using to spam.

Verifying Bingbot
If you see what appears to be Bingbot traffic in your server logs based on a user agent string, for example Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm), and you want to know if this traffic really is originating from a Bing server, you can take the following steps:

Perform a reverse DNS lookup using the IP address from the logs to verify that it resolves to a name that end with search.msn.com
Do a forward DNS lookup using the name from step 1 to confirm that it resolves back to the same IP address
[bing.com...]

[Dimitri]

Verifying Bingbot

Indeed, but I also see lot of legitimate Bingbot requests from other ranges, without reverse.
[webmasterworld.com...]

[oleotio]

Indeed, but I also see lot of legitimate Bingbot requests from other ranges, without reverse.

If the iP doesn't respect robots.txt or has harmful activity, I block it even if it's Google.

On the other hand it is impossible to prevent them from copying content, when you realize and block the damage is already done.

[aristotle]

Trying to identify and block all unwelcome bots can be time-consuming unpleasant work. No matter how many you block, new ones will continue to appear. So my suggestion is to ignore most of them, and only block:

1. Anything that's coming often enough and using enough resources to slow down your website.

2. Anything that's eating enough bandwidth to cost you money.

But from what I've seen, both of those cases are very rare.

[NickMNS]

Indeed, but I also see lot of legitimate Bingbot requests from other ranges, without reverse.

Same here.

I'm seeing a lot of activity from 40.77.[188/189/190/191]. All of these IP's show up in my logs as

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b

Using the tools linked to above to verify, the IP's are not shown to be "Bingbot". Most likely because they are not "Bingbot" in a strict sense but remain a bot operated by Bing as can be seen in the link below:
[bing.com...]

Yes I concede that it is possible that the UA is being spoofed, but I'm using Cloudflare to block the images that are being requested for IP's. CF blocks requests from IP's that are not on it's white-list, and I am not seeing this IP being blocked by CF.

[tangor]

Last few months bingbot has been very aggressive ... from bing IPs ... with a handful of UAs they have announced/published... at two to three times the g hits. Unfortunately, most (over half) are for urls NOT on my site ... which is kind of confusing even though I am returning 403 or 404 for all the noise. That said, g is doing the SAME THING which makes me wonder where these SEs are getting their link profiles from in the first place!