Google Wireless Transcoder

Is this a cell phone user agent?

[grandma genie]

I am seeing all kinds of user agents for iPhones, iPads, iThis and iThat. This one was unusual in that the IPs kept alternating. Here is a sample:

74.125.75.n - - [01/Oct/2010:16:42:46 -0400] "GET /osc/images/box_heading_l.gif HTTP/1.1" 200 114 "my webpage info" "Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13"
74.125.75.n - - [01/Oct/2010:16:42:46 -0400] "GET /osc/images/box_heading_br_4.gif HTTP/1.1" 200 98 "my webpage info" "Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13"
64.233.172.nn - - [01/Oct/2010:16:42:46 -0400] "GET /osc/images/box_heading_tl_4.gif HTTP/1.1" 200 95 "my webpage info" "Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13"
64.233.172.nn - - [01/Oct/2010:16:42:46 -0400] "GET /osc/images/angel.jpg HTTP/1.1" 200 50100 "my webpage info" "Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13"

I assume the visitor is using a cell phone or some type of take-along device to surf the net. Why would the IP keep changing? It is the same user agent string. Is this typical of cell phone visitors? What do you think of the Google Wireless Transcoder? From what I read online, it is supposed to make the site readable on a wireless device.

Grandma_genie

[wilderness]

gg,
Google and a variety of other SE's offer multiple user tools for different uses and devices.

Many of these IP ranges may simply result is "pesty" traffic (to be polite) as well as presenting a doorway for abuse for most webmasters.

You either accept the traffic (leaving open a doorway for abuse) or deny the IP ranges (and all the tools; closing the door).

Sure some innocents will fall by the wayside, however and generally speaking (IMO), not a large enough quantity to fret about.

Each webmaster must determine what is beneficial or detrimental to their own website (s).

[grandma genie]

I notice many user agents showing those tools, but how do you know if the visitor just picked up the tool accidentally or installed it for a sinister purpose? I wish there was a list of those tools that would alert a webmaster (red flag) for potential problems. I'm seeing lots of Funwebproducts in the logs. Is that bad? And that SIMBAR shows up, too. But I have heard that those things are just a problem for the user, not the visited site. Would you say a visitor with a really long user agent string is a red flag to a webmaster? Like these:

24.192.57.nn - - [02/Oct/2010:05:12:56 -0400] "GET /picture.jpg HTTP/1.1" 200 2754 "mywebsite" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; MindQuizSearchToolbar 1.1; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; AskTbARS/5.8.0.12304)"

68.238.218.nn - - [01/Oct/2010:20:34:06 -0400] "GET icon.gif HTTP/1.1" 200 140 "mywebsite" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; FunWebProducts; SIMBAR={5DA5F240-8897-11DF-962E-00248C9D292E}; GTB6.5; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; AskTB5.6; BO1IE8_v1;ENUS)"

[wilderness]

I notice many user agents showing those tools, but how do you know if the visitor just picked up the tool accidentally or installed it for a sinister purpose?

You make a determination based upon the following:
1) what exactly are they viewing and/or grabbing on your site (s).

2) You know the structure of you site (s) and how visitors utilize that structure.

3) you review your raw logs and compare to 1 & 2 above.

4) Finally, is the visit beneficial or detrimental to the goals of your site (s)?

[wilderness]

I wish there was a list of those tools that would alert a webmaster (red flag) for potential problems.

The archives (i. e., old topics) at Webmaster World are archived by multiple search engines.
You just need to become aware of the options for searching those old pages.

Once gain, the questions that you seem to be asking have been discussed here time and again.
You simply need to review these old discussions and not expect an instant notification of intrusion and a solution. . .and all within the same response or email.

[keyplyr]

I closed that security hole years ago by blocking Google's Transcoder & Translater by UA.

[wilderness]

As did most of us keyplr.

Have you seen Jim's reply to gg here [webmasterworld.com]?
Jim's solution makes exceptions for the ranges, however includes header checks.

[Scarecrow]

Google Wireless Transcoder is now more frequently referred to by its unofficial name, "Google Mobilizer." Several Apple web-browser apps use this Google tool, and refer to it in their browser specs as a "compression" feature. What happens when this compression is enabled in that browser is that the page is sent to Mobilizer, and Google sends it back to the mobile device after rewriting all the links on the page. This rewrite keeps the user locked into Mobilizer as they follow links on the pages they surf.

The problem I have with Mobilizer is that it strips out important functionality from the original page. For example, I use a POST input form, and it appears to me, after a lot of research, that Mobilizer disables POST input forms on all pages that use them. I believe that this is because if they let the POST input form through unaltered, the user would drop out of Mobilizer after submitting the form. And it would be a lot of trouble for Google to capture user-submitted POST information, as compared to GET. The GET forms go through okay.

Here is the user-agent for Mobilizer; this came from 64.233.172.17 but Google also uses other IP addresses for this:

Mozilla/5.0 (en-us) AppleWebKit/525.13 (KHTML, like Gecko; Google Wireless Transcoder) Version/3.1 Safari/525.13

When I detect this on one dynamic page I use, I intercept the interceptor with a decoy page, and explain to the user that if they want to use my site on their device, they either have to turn of compression, or they have to start using our SSL input pages.

Google offers a tool that allows you to see how any page will look after Mobilizer processing: [google.com...]

[dstiles]

Thanks for that, Scarecrow. I always assumed, with no means of checking, that it maintained the web page it was given.

Hey, ho! Off to ban another google "bot". :(

[wilderness]

Hey, ho! Off to ban another google "bot".

dstiles,
FWIW, why waste your time attempting to stay on top of all the potentially new tools Google adds to their portals?
Simply deny their 64.233. range and it will not affect their standard bot crawls.
Of course you may apply "exceptions" to this denial, one of which may consider is their translator, however even the later presents vulnerabilities and potential abuse.

[Scarecrow]

I tried numerous Google static IP addresses from different data centers, and by adding "/gwt/n" to the IP address I was able to get into the Google Wireless Transcoder. In other words, you cannot block GWT (aka "Google Mobilizer") by IP address; you have to look for the user-agent.

By the way, this Mobilizer thing is bigger than I thought. While the half-dozen browsers that use Mobilizer for "compression" are all Apple apps, I spent some time looking for Android apps that did the same thing. I couldn't find any. Now I know why. My Android 2.2 smartphone out of the box, with the standard 2.2 package and not much else, requires a Gmail account and is also tied into Google Mobile Search by default. When you use Google Mobile Search you get web pages without any processing. But there is also a little "Options" drop-down box next to the links, and if you select "Mobile formatted" it uses the exact same Mobilizer engine.

In other words, the Mobilizer engine as a "compression" option in Android browser apps would be rather redundant and pointless. Google already owns that space on Android.

We can all be grateful that Mobilizer processing isn't the default when you click on a link from Google Mobile Search. It requires a couple extra clicks to use it.

[Scarecrow]

You can simulate a Google Mobile Search from Android 2.2 on your desktop by following these steps:

1) Enable Javascript

2) Use a browser that lets you stuff any user-agent you like, and temporarily use this:

Mozilla/5.0 (Linux; U; Android 2.2; en-us; Comet Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

3) Go to the Mobilizer entry page at [google.com...]

4) Do not use the first input box, but go to the bottom to the "Google Search" box. This way you are using Google Mobile Search.

5) The fact that Google believes you are on a handset, due to that user-agent, causes the little Javascript "Options" link to appear. When you click on this, there is a "Mobile formatted" option. Click on that and the web page behind the link is filtered through the Mobilizer before you see it.

[dstiles]

Scarecrow - Hey! How secure and helpful is that little procedure?! :)

Thanks for the info. I'll find some time to run that, even at the risk of accessing a google web site. :(

Wilderness - a lot of google tools such as translater run on various IPs in their ranges. If I blocked the translater I would upset my clients AND their clients.

My solution is to enable specific UAs and monitor them. I never bothered with transcoder before as it seemed innocuous.

[keyplyr]

I never bothered with transcoder before as it seemed innocuous. - dstiles

I agree, however my decision to block Google's Transcoder and Translator comes from endless scraping events by nefarious agents I had previously blocked by IP range, now using these tools to sneak in. I have not seen any negative repercussions from doing so.

[SevenCubed]

I just read through this thread to make sure I'm not overlooking something before I throw this in. Why would you absolutely want to block the transcoder?

I'm not aware of all that it is used for but just last week I was checking out my website through my 1 x 1¼ inch Samsung mobile phone browser through Google mobile search. If I had blocked them I would not have been able to access my site as I did. Going through that interface gave me access to the site that is otherwise meant to be accessed by desktop devices but is WAP compatible because of the xhtml markup. By design I have made it easy for interfaces like the Google transcoder to work with my site thereby making it accessible to devices like my out dated mobile phone. It displayed exactly like the desktop version -- pixel for pixel.

Essentially if you block them you are blocking potential visitors. Am I not understanding something here? If yes, please point out what I am missing.

[wilderness]

Why would you absolutely want to block the transcoder?

Because you offer content which exceeds the capabilities of a 1 x 1¼ screen, like a 1,500-3,000 word count page with images added.

[SevenCubed]

I have high word counts and large images to boot -- it displays perfectly on a tiny screen at an adjusted magnification. The transcoder then gives an option to zoom in to selected areas they map out so I can actually read the text. After reading it I can zoom back out to take in the whole page again.

[keyplyr]

Why would you absolutely want to block the transcoder?

I'm not aware of all that it is used for but just last week I was checking out my website through my 1 x 1¼ inch Samsung mobile phone browser through Google mobile search. If I had blocked them I would not have been able to access my site as I did.

Well, as I said above... scrapers, thieves, unwanted agents have used these Google tools to get around the IP blocks I have for them. They are a huge security hole IMO.

AFAIK only older mobile browsers need the Transcoder to help rendering. Recent builds of Android, Blackberry, Windows Mobile and Palm all render my web pages just fine and I've had the transcoder UA blocked for well over 2 years. Google Mobile Search also ranks my sites just fine.

[SevenCubed]

I guess they must be very determined to get your content if they resort to trying to sneak in like that.

I know that the modern hand-helds don't need this. But by knowing that my site can display in one of the most out-dated devices on the market I know it can be viewed in anything.

In any case, I just wanted to point out that the Google Transcoder UA is not all bad, it has it's place.

[wilderness]

I have high word counts and large images to boot -- it displays perfectly on a tiny screen

And does the bot make make multiple requests for the same page to accomplish this (as did many older small dispaly machines (similar to the viewing of a multi-page PDF file) or, does the bot require a cache of the page to accomlish this?

In any event 3,000 words on such a small device must be many pages?

[SevenCubed]

I'm not sure about the first part of your question. I'd have to go unzip some log files from some day last week to find out. Even then I'm not sure what day it was. If would be of a useful purpose for you I can do it, otherwise I'll just say I don't know. But thinking about it a bit, it wouldn't have been a cached version because I did go examine the log files afterward to see how it looked from the inside out.

As far as how many pages -- the interface gives me a reduced magnification where I can see the whole page all on one tiny screen, it leaves nothing out. From within that view it places numbered tags so that I can jump to key navigation areas using accesskey functionality. It saves having to scroll the entire page at once.

At the end of key areas it then provides an option to zoom back out again for a full page view. I had no complaints about the quality or speed. So the actual number of pages was predefined by Google according to primary and secondary navigation by those tagged areas. Hope I've explained it well enough.

[wilderness]

Many thanks.

Please don't pursue anything beyond recollection.