Welcome to WebmasterWorld Guest from 54.158.248.167

Forum Moderators: Ocean10000 & incrediBILL & phranque

Ip address keeps visiting my Apache server

     
1:51 pm on Jul 18, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


So this ip address [snip] visits my apache server every few minutes.

Obviously I blocked it and now it keeps on going. Is there any page to report this ip address or further blocking?

I'm using AWS instance with apache server in it. Thanks!

[edited by: phranque at 11:13 pm (utc) on Jul 18, 2017]
[edit reason] see forum Charter [/edit]

2:29 pm on July 18, 2017 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:3451
votes: 181


How did you block it? I mean what are you using to block it with?
3:05 pm on July 18, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


I am behind Cloudflare so I blocked the ip from their firewall settings.
3:27 pm on July 18, 2017 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:3451
votes: 181


If the CIDR is blocked, they may still appear in your logs. They should be receiving 403 responses rather than 200. You can check that in the access logs.
8:08 pm on July 18, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:9641
votes: 482


As not2easy says, blocking does not stop the agent from attempts to access your files, it stops successful attempts.

These unsuccessful attempts still show in your raw server logs along with other communications.
10:21 pm on July 18, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts:141
votes: 12


Send a note to [the hosting company's] Security and complain about [the Contact]. I've not complained to them in the past, so I don't know. You might get lucky. I find complaining to ISPs is a 50-50% success rate. Some ISPs will not do anything. Of course ISPs do not like when you ban their ranges, so have some incentive to do something.

[edited by: phranque at 7:19 am (utc) on Jul 19, 2017]
[edit reason] see forum Charter [/edit]

11:25 pm on July 18, 2017 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10980
votes: 84


I blocked the ip from their firewall settings

unsuccessful attempts still show in your raw server logs along with other communications

if it's blocked at the firewall, it shouldn't get through to the web server - blocked IP addresses wouldn't appear in the web server log.
11:36 pm on July 18, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14028
votes: 521


Send a note to [the ISP's] Security

I don't think it is an ISP. I looked it up before the thread was edited, and I've got it down as a server range. Good luck blocking those, unless the robot is doing something over-the-top destructive like, say, making requests every few nanoseconds rather than every few minutes.

born2run, I don't think you ever said (pre-edit) what the visitor is requesting. Just the root, or something more interesting and/or complicated?
11:48 pm on July 18, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:9641
votes: 482


I don't think it is an ISP. I looked it up before the thread was edited, and I've got it down as a server range.
Yes, the IP belongs to Datashack, a large server farm.

Datashack ranges are listed here: [webmasterworld.com...]
2:30 am on July 19, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


So here's the details, it's using different UAs like:

(compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)
(compatible; YandexImages/3.0; +http://yandex.com/bots)

Visits like every one minute at a time. I guess it's a bot that's not stopping.
2:35 am on July 19, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:9641
votes: 482


born2run - these are common User Agents [webmasterworld.com] coming from common Server Farm Ranges [webmasterworld.com]

The information is here in the forums. You need to do some reading. Most all your questions have already been discussed, some many many times. Use the Search Utility in the upper-right corner.
2:42 am on July 19, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts:141
votes: 12


(compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)
(compatible; YandexImages/3.0; +http://yandex.com/bots)

These are completely different bots. In my .htaccess here's how I ban them:

SetEnvIf User-Agent AhrefsBot keep_out
order allow,deny
allow from all
deny from env=keep_out

I allow Yandex to index my site. There are a couple of different Yandex bots. They are not too bad.
2:47 am on July 19, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:9641
votes: 482


Both of those bots obey robots.txt

No need to block, just disallow.
8:50 am on July 19, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


My point is that from one IP address two useragents are being seen. So it's a fake bot I presume
9:01 am on July 19, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:9641
votes: 482


Well that's certainly a possibility. The Yandexbot comes from verifiable Yandex ranges.

The AhrefsBot used to come from Choopa ranges but could have moved to Datashack.

It really helps to start keeping notes on these agents, what they do, where they come from. After a while it all starts to make sense.

Also, since CDNs use multiple file servers from various points around the world, the log reporting can be unreliable. Things get changed around so it isn't consistent.... but you probably have found that out by now.
12:47 pm on July 19, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


Thanks! ok I have another issue. lots of visitors being "challenged" by Cloudflare coz of this reason:

False IE6 detection [Type C]

What's this about anyways?
5:15 pm on July 19, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14028
votes: 521


False IE6 detection [Type C]
Heh. My host has a mod_security option, and one pattern they block is a UA string involving IE6. Wouldn't be surprised if Cloudflare looks for similar patterns.

:: shuffling papers ::

I think it's this, based on searching logs for 418 response + MSIE 6:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

Idle query: When did anyone last detect a non-false IE6? I don't think I've seen one since 2014.
1:01 pm on July 21, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


Ok great thanks all, I'll have to monitor traffic every few hours to check what's been going wrong. Thanks again!
1:25 pm on July 31, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


Yeah I complained to the owner of that IP address. So let's see in a few days.
4:14 pm on July 31, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 141
votes: 12


I usually file a complaint with the IP address' host provider, not the actual IP address owner. I find that I receive no reply from IP address owners. The host provider can put pressure on the owner to change or get kicked off their host.
12:08 am on Aug 1, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:658
votes: 7


Yep I emailed the host provider too. No reply so far, and the ip is still visiting my site :-(
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members