Forum Moderators: open

Message Too Old, No Replies

crawl-66-249-66-66.googlebot.com

How?

         

blend27

5:06 pm on Nov 26, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Aint that a Turkey?

And his 91.208.197.96 translates to this crawl-66-249-66-66.googlebot.com

ALEXHOST S.R.L(Moldova) - AS200019
91.208.197.0/24

I checked a few site site like [abuseipdb.com...] and and [bgp.he.net...] for sure DNS is listed as googlebot.

jmccormac

5:19 pm on Nov 26, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Reverse DNS set up to spoof Googlebot.

Regards...jmcc

not2easy

6:10 pm on Nov 26, 2021 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Sometimes learning something new is not what you want. :(

jmccormac

6:17 pm on Nov 26, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yep. Happy Thanksgiving. :)

It might be safer to use IP level whitelisting for Googlebot and Bing in httpd.conf.

Regards...jmcc

tangor

9:32 am on Nov 27, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



BIG fan of whitelisting the important players. Makes it so much easier to kick the fakes to the curb. :)

blend27

12:07 pm on Nov 27, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



..this one got caught:

ip: 91.208.197.96
time: {ts '2021-11-25 04:57:51'}
method: GET
protocol: HTTP/1.1
connection: close
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:68.0) Gecko/20100101 Firefox/68.0
Accept-Encoding: gzip
content-length: 0

--------------------Plus------------------------------
:missing headers
:invalid header values
:outdated ua
:spoofed dns
:hosting ip range
:country is not is not on the list of normal visitor origination
:ip range is only /24

jmccormac

1:08 pm on Nov 27, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It seemed to be checking index pages for websites.

Regards...jmcc

lucy24

6:00 pm on Nov 27, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Sometimes checking rDNS is just overkill. If it claims to be Googlebot but isn’t from 66.249.blahblah, slam the door in their face and don’t listen to any impassioned pleading from behind the door.