Google Wireless Transcoder

[dstiles]

For those who value hits from this mobile proxy and who also include hidden bot traps on pages, be aware of the following:

A link hidden by CSS becomes visible at default font height (assuming there is something within the link's text). CSS, even in-line, is ignored. Sod's law says this link WILL be clicked on! :(

A 403 rejection is reported by the proxy as 502 (Bad Gateway) - innacurate although probably reasonable.

The IP is Google's with the visitor's IP in the proxy's Forwarded_For. Two Google ranges at two accesses a few hours apart:

209.85.170.nnn
216.239.50.nnn

and the original access from a visitor that triggered the investigation:

209.85.136.nnn

The proxy's Via header is (at present)
1.0 proxy.google.com:80 (squid)

I tested using Firefox on Windows but the UA seems to always be:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Google Wireless Transcoder;)

[keyplyr]

GWT will also send requests with no UA string at all.

[Samizdata]

When requesting images (a checkbox option) the IP often changes and the UA can be:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;Google Wireless Transcoder;)

Note the lack of a space before the word "Google".

For testing purposes you can use [google.com...]

...

[dstiles]

Thanks. That should still work with the trap bypass I devised.

If the GWT UA is blank then all bets are off. I don't permit blank UAs.

[keyplyr]

...I don't permit blank UAs

I don't either, but keep an IP white list. Google mobile will also request flat files without a UA as well as one of the MS utilities (forget which one.)

[dstiles]

Frankly I'm fed up pandering to dumb search engines, especially google.

I go a long way to let SEs access my clients' sites (my own sites are another matter) but if they can't respect me they don't get respect back. Google should know better. Their multi-purpose use of rDNS-less proxies is abysmal and if they choose to use stupid UAs (or none) then they're likely to be rejected.