Welcome to WebmasterWorld Guest from 54.90.204.233

Forum Moderators: Ocean10000

Message Too Old, No Replies

Google Test-Bot: Google-Test2

     
11:36 am on Dec 18, 2013 (gmt 0)

Senior Member from DE 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2003
posts:890
votes: 6


I saw a new bot from Google with the name Google-Test2. From the IP 72.14.199.91 one can see that this is indeed from Google.

The bot doesn't respect the rules of the robots.txt file which allows access for GoogleBot but not for this test bot.

The URLs are really strange:
http://www.example.vom/page.html%3C/web:Url%3E%3Cweb:DisplayUrl%3Ewww. [...]

These are url-encoded versions of a XML file of the form


<web:Url>
<web:DisplayUrl>www.example.vom/page.html</web:DisplayUrl>
<web:DateTime>2012-01-12T01:54:00Z<web:DateTime>
[...]


Anyone else is seeing this bot or having an idea what it is good for?
3:14 pm on Dec 18, 2013 (gmt 0)

Senior Member from DE 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2003
posts:890
votes: 6


I forgot to mention that the HTTP referer
http://www.google.co.uk/
seems to to a fake.
3:48 pm on Dec 18, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5496
votes: 3


Here's a 2010 thread [webmasterworld.com] with a resolution in the last submission by Jim.

However and considering the recent insight provided by dstiles, you could reduce

RewriteCond %{REMOTE_ADDR} ^66\.249\.(6[4-9]|7[0-9]|8[0-46-9]|9[0-5])\. [OR]

to

RewriteCond %{REMOTE_ADDR} ^66\.249\.(6[4-9]|7[0-9])\. [OR]
4:40 pm on Dec 18, 2013 (gmt 0)

Senior Member from DE 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2003
posts:890
votes: 6


I'm not sure if I got your point...

I don't wat to block this bot (which can be easily done by blocking the user-agent 'Google-Test2').

If have several fake GoogleBots which I'm already blocking. However, this is a real Google bot because the IP 72.14.199.91 belongs to Google. The same user-agent can be found on other logfiles [google.com] and similar problems with this kind of URLs [productforums.google.com] can be found.

I'm just curious about what is causing these URLs and what this bot is good for.
4:49 pm on Dec 18, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5496
votes: 3


72.14.204.136 - - [27/Jul/2008:09:27:15 -0500] "GET /MyFolder/MySub/Sub-Sub/MyPage.html HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Google Wireless Transcoder;)"
72.14.204.136 - - [31/Jul/2008:18:32:49 -0500] "GET /MyFolder/MySub/MyPage.html HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Google Wireless Transcoder;)"


Web Accerelator 2007 [webmasterworld.com]

72.14.194.27 - - [21/Oct/2006:11:27:09 -0700] "GET / HTTP/1.1" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0)"
4:52 pm on Dec 18, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5496
votes: 3


FWIW, there are three currently active Google threads.

Please review "Google is that You?"
10:40 pm on Dec 18, 2013 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3225
votes: 17


doc_z - you are missing something there. The IP 72.14.199.91 is NOT a google bot IP. It's in a bannable range (see my comment in another current G thread). Its precise rDNS entry cl;aims it is a rate-limited proxy, which means it may well not be G at all but some criminal using their proxy.

"Real" googlebots ONLY come from IPs labelled in DNS as crawler bots.

Block: 72.14.192.0 - 72.14.255.255
1:09 am on Dec 19, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15450
votes: 739


You may need to poke holes though.

72.14.199 includes Site Verification (for wmt)
72.14.229 includes humans investigating dmca claims
4:03 pm on Dec 19, 2013 (gmt 0)

Senior Member from DE 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2003
posts:890
votes: 6


Okay, I got it. I thought it's Google because the IP is from Google.

Btw, I saw the same bot with the IP 209.85.238.208.
6:31 pm on Dec 19, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5496
votes: 3


You nay add 209.85.128.0/17 to your denies as well.
8:28 pm on Dec 19, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15450
votes: 739


because the IP is from Google

"belongs to {name}" != "{name} indexing robot"

Some search engines are better than others at preserving the distinction.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members