Welcome to WebmasterWorld Guest from 54.224.83.221

Forum Moderators: Ocean10000 & keyplyr

Buck

     
7:49 pm on Mar 17, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


UA: Buck/2.2 (Googlebot/2.1; +https://app.hypefactors.com/media-monitoring/about.html)
Protocol: HTTP/1.1
Robots.txt: Yes
Host: AWS
35.152.0.0 - 35.183.255.255
35.176.0.0/13, 35.160.0.0/12, 35.152.0.0/13

They will soon learn that pretending to be Googlebot will not serve them well.
9:23 pm on Mar 19, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


Since this agent requests robots.txt, I added it to my disallow list and gave it a few days to process. It continues to ignore the disallow.
1:59 pm on Mar 20, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1960
votes: 62


What the BUCK do the mean Googlebot/2.1 any-how? Why try to get a pass as one?
8:56 pm on Mar 20, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


A lot of bot runners somehow think they'll get past block filters if they include Googlebot in their UA string. IMO this is self-destructive and just the opposite of how most site owners filter.

For example, I block anything calling itself Googlebot, allowing only those from valid Googlebot crawl IP ranges. So the above UA was actually blocked because it used Googlebot... however, it was also blocked because the range was AWS. While I currently allow a couple dozen UAs from AWS ranges, this is not one of them.
9:47 pm on Mar 22, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


Now coming from...

Host: AWS
52.84.0.0 - 52.95.255.255
52.84.0.0/14, 52.88.0.0/13

Still requesting robots.txt, then ignoring it.
11:03 pm on Apr 19, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


New UA: Buck/2.2; (+https://app.hypefactors.com/media-monitoring/about.html)

Still coming from various AWS IP ranges & disobeying robots.txt (where it is disallowed.)

If you want Buck to stop monitoring your site, you can simply configure your robots.txt accordingly.
This has not been my experience.
1:55 am on Apr 25, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:847
votes: 10


Yes I see it too on my site. I blocked the UA. It's crazy these new companies running bots,
9:52 pm on July 10, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


Also coming from...

Host: AWS
18.208.0.0 - 18.208.255.255
18.208.0.0/16

which may be a new AWS range for some
10:50 pm on July 13, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1960
votes: 62


Haven't seen that range yet, but they(AWS) keep grabbing /16's, don't they....
10:59 pm on July 13, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


(AWS) keep grabbing /16's, don't they....
And /15's and /14's...

AWS
18.202.0.0 - 18.207.255.255
18.204.0.0/14, 18.202.0.0/15
1:02 am on July 14, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14905
votes: 649


Dig up an old IANA list and you'll find huge numbers of /8s allocated to single corporations (Microsoft, poor dears, missed the land grab) and governmental entities (not just US govt but all over the place). Bit by bit, almost all of these are now getting apportioned elsewhere as their owners realize they don't actually need 2^24 external IP addresses ... and what the previous owners don't want, AWS does.
1:26 am on July 14, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


Dig up an old IANA list and you'll find huge numbers of /8s allocated...
And Buck/2.2 is coming from them?
2:58 am on July 14, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14905
votes: 649


Not necessarily; I was referring to the “AWS keeps grabbing” earlier. Many of those new-to-us AWS ranges are IPs we never set eyes on until recently, because they were allocated to single-users who, well, didn't use them. I first noticed it with Merck some years back, and now plenty of others are following suit.
8:23 am on July 14, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Nov 13, 2016
posts:463
votes: 65


While I currently allow a couple dozen UAs from AWS ranges, this is not one of them.

Is there a topic where you had posted these UAs (from AWS) that you are authorizing? I'd like to study them, if you don't mind.
8:29 am on July 14, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12083
votes: 770


Sorry Dimitri,

For the integrity of the forum, it is policy not to label any UA or IP (range) as good or bad.

All we do here is document unbiased information, and from that, each site owner can decide whether it is of benefit to their interests.

I can say that every UA coming from AWS ranges that I allow, are documented in these forums.

[fix typo]

[edited by: keyplyr at 8:35 am (utc) on Jul 14, 2018]

8:32 am on July 14, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Nov 13, 2016
posts:463
votes: 65


ok