Welcome to WebmasterWorld Guest from 107.20.54.98

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

strange googlebot referer

googlebot referral from odd site

     
3:39 pm on Aug 3, 2014 (gmt 0)

New User

joined:Apr 25, 2012
posts: 40
votes: 0


66.249.79.3 - - [02/Aug/2014:18:51:55 -0600] "GET / HTTP/1.1" 200 11420 "http://fr.wschange.com/new/2013/09/12/wildplum.biz/redirect" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"


never seen this before. is there some relationship between google and fr.wschange.com?
7:35 pm on Aug 3, 2014 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts: 2038
votes: 1


It's rare to see detailed G referrers nowadays and I don't know how to explain them when I do. Back in May I made a note of seeing --

crawl-66-249-66-14.googlebot.com
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

-- with a StumbleUpon ref:

http://www.stumbleupon.com/su/123ABC/www.mysite.com/dir/filename.html

Beats me. I just figure oddities happen.
10:18 pm on Aug 3, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12719
votes: 244


It's rare to see detailed G referrers nowadays

For page files, that is. For some types of non-page files, notably js, a referer is very common. But the referer will then be your own page.

:: detour to check ::

Nope, can't find any at all. (Real googlebot, I mean. There's some Ukrainian referer spam, even if I constrain the search to 200 responses.)
2:50 am on Aug 27, 2014 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


I see googlebot leaving referrers quite frequently, sometimes using a googlebot user agent and other times using browser agents. Here's one that hit twice on the 11th. Each time a different referrer was used:

46.165.231.151
crawl-46-165-231-151.googlebot.com
Mon Aug 11 22:46:22 2014
HTTP_ACCEPT{'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'}
HTTP_ACCEPT_ENCODING{'gzip, deflate'}
HTTP_ACCEPT_LANGUAGE{'ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3'}
HTTP_CONNECTION{'close'}
HTTP_HOST{'www.example.com'}
HTTP_REFERER{'http://www.banffspringsgolf.com/'}
HTTP_USER_AGENT{'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0'}

That might be an unknown googlebot ip range to WebmasterWorld users.
3:36 am on Aug 27, 2014 (gmt 0)

Moderator from US 

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:2571
votes: 48


Um, that's because WHOIS says it belongs to Leaseweb/NETDIRECT.
4:46 am on Aug 27, 2014 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


That is not unusual. Google leases ip allocations from a number of internet service providers located all around the world.
6:16 am on Aug 27, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:5817
votes: 64



That is not unusual. Google leases ip allocations from a number of internet service providers located all around the world.

Well I suppose there is the remote possibility that is accurate... I don't work at Google so I don't know for sure :)

However the most likely scenario (and one I feel *is* accurate) is the hit from the Leaseweb Server Farm is a malicious actor spoofing as Googlebot. I get about 20 to 50 of these spoof attempts from Leaseweb per month (and thousands from other server farms and infected machines) and I say "attempts" because I block them all.
7:05 am on Aug 27, 2014 (gmt 0)

Junior Member

joined:May 3, 2011
posts:75
votes: 0


How to verify Googlebot:

[googlewebmastercentral.blogspot.com...]

[edited by: MickeyRoush at 7:06 am (utc) on Aug 27, 2014]

7:06 am on Aug 27, 2014 (gmt 0)

Full Member

5+ Year Member

joined:Aug 16, 2010
posts:214
votes: 11


Never trust the PTR record of IP address. It can be fake.

So double check: IP -> PTR record and then PTR -> IP address.

crawl-46-165-231-151.googlebot.com is not resolving to 46.165.231.151 so it is fake.