homepage Welcome to WebmasterWorld Guest from 54.211.7.174
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
AppEngine-Google: proxy1china
Umbra




msg:4159182
 1:20 pm on Jun 25, 2010 (gmt 0)

72.14.192.1
AppEngine-Google; (+http://code.google.com/appengine; appid: proxy1china)

Looks like an app to circumvent Chinese censorship, fair enough. However, for that exact same IP address, I also get user agents that include:

Google Wireless Transcoder
Google Keyword Generator
(via translate.google.com)
Google-Site-Verification
blank UA/referer
and more...

Beforehand, I hesitated to ban non-Googlebot IPs like this one, because of legitimate uses like Google Site Verification, translation tools, etc. But now Google is sharing their IP address with 3rd parties? Are they insane?

The madness must stop. I have dared to ban my first Google IP address.

Somebody tell Google! One IP address = One tool ONLY!

 

keyplyr




msg:4159317
 4:25 pm on Jun 25, 2010 (gmt 0)

I allow all Google IPs for that same reason, but ban specific UAs; Transcoder & Translator among them.

dstiles




msg:4159447
 7:28 pm on Jun 25, 2010 (gmt 0)

Google has always flouted its "we name all bots with rDNS" rule. Some of their IPs are public proxies as well as pseudo-bots. Considering the number of IPs they own it's a ludicrous situation engendered, I suspect, by laziness.

I allow bot company ranges (eg google that isn't googlebot, yahoo that isn't slurp) on an expiry basis: it gets blocked for doing something I deem "illegal" for a period of time then I let it play again, the period being extended every time it mis-behaves.

As a result I probably block a lot of transcoders because they do not use the correct IP or because some proxy-user has previously abused it. I've given up worrying: it's only google being its usual stupid self, after all.

Umbra




msg:4164753
 2:14 pm on Jul 5, 2010 (gmt 0)

I recently caught 74.125.74.196 via translate.google.com hitting a webpage associated with Honeypots and scrapers/harvestor activity. Another exploitable IP courtesy of Google.

Umbra




msg:4164769
 2:32 pm on Jul 5, 2010 (gmt 0)

I recently caught 74.125.74.196 via translate.google.com hitting a webpage associated with Honeypots and scrapers/harvestor activity. Another exploitable IP courtesy of Google.
Also 74.125.75.19
Pfui




msg:4164909
 7:36 pm on Jul 5, 2010 (gmt 0)

(Maybe this thread could be retitled 'Google's Atypical Apps' or some such? Anyway...)

The OP mentions IP 72.14.192.1. Here's one-off, from December, 2009:

72.14.192.2
Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13,Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

FWIW, a different site. Today:

TIME: 11:30:27 - First hit, pre-redirect :

74.125.152.80
Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

TIME: 11:30:28 - One second later, redirected hit -- with a new IP and a 'new' UA(!):

72.14.202.83
Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13,Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

Note the long UA is malformed. It repeats itself; plus there's no space after the comma when it does (525.13,Mozilla).

Umbra




msg:4165186
 1:26 pm on Jul 6, 2010 (gmt 0)

Note the long UA is malformed. It repeats itself; plus there's no space after the comma when it does (525.13,Mozilla).
I've also just seen that malformed UA from:
64.233.172.18 -- also related to appengine [webmasterworld.com]
66.249.82.2 -- also related to translate.google.com

Umbra




msg:4169213
 1:13 pm on Jul 13, 2010 (gmt 0)

Considering the number of IPs they own it's a ludicrous situation engendered, I suspect, by laziness
I came across a discussion in which an IT guy needed to configure their office firewall to allow access to Google Translator except that the IP address kept changing. A Google employee replied that they use a "relatively small" set of IP addresses for "various load balancing reasons" and that the IT guy should manually create and maintain their own "small set" of Google translator IP addresses.
dstiles




msg:4169517
 9:16 pm on Jul 13, 2010 (gmt 0)

Fine, except the IPs are also used as proxies, bots etc.

If they can't be bothered to allocate proper rDNS and keep the IPs uni-purpose then, to be polite about it, stuff 'em. Translate is often abused anyway, as far as I can tell.

Umbra




msg:4175783
 2:40 pm on Jul 24, 2010 (gmt 0)

74.125.154.86
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 AppEngine-Google; (+http://code.google.com/appengine; appid: ascraper)

The app id is actually "ascraper" ... Google, you have got to be kidding me!

Umbra




msg:4203624
 1:14 pm on Sep 18, 2010 (gmt 0)

64.233.172.18
Wink Client v1.1.1 AppEngine-Google; (+http://code.google.com/appengine; appid: lakerda22)

IP address 64.233.172.18 is also shared with Google Wireless Transcoder, so yet another questionable Google tool in bed with a 3rd party app

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved