homepage Welcome to WebmasterWorld Guest from 54.204.168.212
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
AppEngine-Google: proxy1china
Umbra

10+ Year Member



 
Msg#: 4159180 posted 1:20 pm on Jun 25, 2010 (gmt 0)

72.14.192.1
AppEngine-Google; (+http://code.google.com/appengine; appid: proxy1china)

Looks like an app to circumvent Chinese censorship, fair enough. However, for that exact same IP address, I also get user agents that include:

Google Wireless Transcoder
Google Keyword Generator
(via translate.google.com)
Google-Site-Verification
blank UA/referer
and more...

Beforehand, I hesitated to ban non-Googlebot IPs like this one, because of legitimate uses like Google Site Verification, translation tools, etc. But now Google is sharing their IP address with 3rd parties? Are they insane?

The madness must stop. I have dared to ban my first Google IP address.

Somebody tell Google! One IP address = One tool ONLY!

 

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4159180 posted 4:25 pm on Jun 25, 2010 (gmt 0)

I allow all Google IPs for that same reason, but ban specific UAs; Transcoder & Translator among them.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4159180 posted 7:28 pm on Jun 25, 2010 (gmt 0)

Google has always flouted its "we name all bots with rDNS" rule. Some of their IPs are public proxies as well as pseudo-bots. Considering the number of IPs they own it's a ludicrous situation engendered, I suspect, by laziness.

I allow bot company ranges (eg google that isn't googlebot, yahoo that isn't slurp) on an expiry basis: it gets blocked for doing something I deem "illegal" for a period of time then I let it play again, the period being extended every time it mis-behaves.

As a result I probably block a lot of transcoders because they do not use the correct IP or because some proxy-user has previously abused it. I've given up worrying: it's only google being its usual stupid self, after all.

Umbra

10+ Year Member



 
Msg#: 4159180 posted 2:14 pm on Jul 5, 2010 (gmt 0)

I recently caught 74.125.74.196 via translate.google.com hitting a webpage associated with Honeypots and scrapers/harvestor activity. Another exploitable IP courtesy of Google.

Umbra

10+ Year Member



 
Msg#: 4159180 posted 2:32 pm on Jul 5, 2010 (gmt 0)

I recently caught 74.125.74.196 via translate.google.com hitting a webpage associated with Honeypots and scrapers/harvestor activity. Another exploitable IP courtesy of Google.
Also 74.125.75.19
Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4159180 posted 7:36 pm on Jul 5, 2010 (gmt 0)

(Maybe this thread could be retitled 'Google's Atypical Apps' or some such? Anyway...)

The OP mentions IP 72.14.192.1. Here's one-off, from December, 2009:

72.14.192.2
Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13,Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

FWIW, a different site. Today:

TIME: 11:30:27 - First hit, pre-redirect :

74.125.152.80
Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

TIME: 11:30:28 - One second later, redirected hit -- with a new IP and a 'new' UA(!):

72.14.202.83
Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13,Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

Note the long UA is malformed. It repeats itself; plus there's no space after the comma when it does (525.13,Mozilla).

Umbra

10+ Year Member



 
Msg#: 4159180 posted 1:26 pm on Jul 6, 2010 (gmt 0)

Note the long UA is malformed. It repeats itself; plus there's no space after the comma when it does (525.13,Mozilla).
I've also just seen that malformed UA from:
64.233.172.18 -- also related to appengine [webmasterworld.com]
66.249.82.2 -- also related to translate.google.com

Umbra

10+ Year Member



 
Msg#: 4159180 posted 1:13 pm on Jul 13, 2010 (gmt 0)

Considering the number of IPs they own it's a ludicrous situation engendered, I suspect, by laziness
I came across a discussion in which an IT guy needed to configure their office firewall to allow access to Google Translator except that the IP address kept changing. A Google employee replied that they use a "relatively small" set of IP addresses for "various load balancing reasons" and that the IT guy should manually create and maintain their own "small set" of Google translator IP addresses.
dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4159180 posted 9:16 pm on Jul 13, 2010 (gmt 0)

Fine, except the IPs are also used as proxies, bots etc.

If they can't be bothered to allocate proper rDNS and keep the IPs uni-purpose then, to be polite about it, stuff 'em. Translate is often abused anyway, as far as I can tell.

Umbra

10+ Year Member



 
Msg#: 4159180 posted 2:40 pm on Jul 24, 2010 (gmt 0)

74.125.154.86
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 AppEngine-Google; (+http://code.google.com/appengine; appid: ascraper)

The app id is actually "ascraper" ... Google, you have got to be kidding me!

Umbra

10+ Year Member



 
Msg#: 4159180 posted 1:14 pm on Sep 18, 2010 (gmt 0)

64.233.172.18
Wink Client v1.1.1 AppEngine-Google; (+http://code.google.com/appengine; appid: lakerda22)

IP address 64.233.172.18 is also shared with Google Wireless Transcoder, so yet another questionable Google tool in bed with a 3rd party app

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved