New Googlebot

Nokia6820/2.0

[Mokita]

Full UA: Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

coming from 66.249.65.129

I've never seen this in our sites before. I guess it is pretending to be a WAP phone.

I'm wondering why they need a different bot for this? What does it see differently to the normal googlebot?

[volatilegx]

Just saw it here, too, coming from 66.249.66.1

[Pfui]

I've been wary of that bot and other Google mobiles/WAP phones (Wireless Application Protocol) since early last month when one of its ilk didn't ask for robots.txt and then proceeded to go every which way, hitting my bot traps along the way.

'Twas this UA, from 64.233.166.136, almost like the one you saw:

Nokia3510i/1.0 (05.30) Profile/MIDP-1.0 Configuration/CLDC-1.0 (Google WAP Proxy/1.0)

(Saw that last Nov., too, again from 64.233.166.136; again no robots.txt.)

FWIW...

I'm also not wild about the following info excerpt, emphasis mine, but have yet to specifically opt-out, preferring instead to rewrite G's WAP-related UAs* until the jury's back on whether they routinely ask for, let alone heed robots.txt.

-----
Source: Google Information for Webmasters
Need to remove content from Google's index? [google.com]

[...]

Remove transcoded pages [google.com]

Google Web Search on mobile phones allows users to search all the content in the Google index for desktop web browsers. Because this content isn't written specifically for mobile phones and devices and thus might not display properly, Google automatically translates (or "transcodes") these pages by analyzing the original HTML code and converting it to a mobile-ready format. To ensure that the highest quality and most useable web page is displayed on your mobile phone or device, Google may resize, adjust, or convert images, text formatting and/or certain aspects of web page functionality.

To prevent your web page(s) from being transcoded, please send a removal request to [e-mail].
-----

Thoughts? About the WAPs? About the transcoding? (The latter makes G's caching look like a walk in the park, copyright-wise.)

.
*Google WAPs

MOT-V171 UP.Browser/6.2.2.7 (GUI) MMP/1.0 UP.Link/6.3.0.0.0 (Google WAP Proxy/1.0)

Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

Nokia3510i/1.0 (05.30) Profile/MIDP-1.0 Configuration/CLDC-1.0 (Google WAP Proxy/1.0)

Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 (compatible; Googlebot/2.1; [google.com...]

[thetrasher]

[webmasterworld.com ]

pfui: "Google WAP Proxy/1.0" is a proxy, not a googlebot.

[volatilegx]

Right. I do not believe this bot is using WAP proxy IP addresses.

[Pfui]

Here's my, um, 'rogue robot' philosophy...

"If it looks like a bot, and acts like a bot, and hits bot traps -- it's banned."

: )

[Mokita]

I found this information, which confirms it is definitely a bot, similar to Googlebot-Image:

If you go to the newly added robots.txt tab in Google Sitemaps, at the very bottom there is a drop down list. Look inside that, and you'll see this:

Googlebot-Mobile : crawls pages for our mobile index

So I guess it can be banned via robots.txt if required. Presumably (?) it will request and obey it.

[Pfui]

From robots.txt:

User-agent: Googlebot-Mobile
Disallow: /

Alas, didn't matter:

crawl-66-249-66-133.googlebot.com
[27/Feb/2006:19:25:25 -0800] "GET /robots.txt
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

crawl-66-249-66-133.googlebot.com
[27/Feb/2006:19:25:26 -0800] "GET /
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

crawl-66-249-66-133.googlebot.com
[28/Feb/2006:01:47:36 -0800] "GET /welcome.html
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

crawl-66-249-66-133.googlebot.com
[28/Feb/2006:01:48:45 -0800] "GET /dir/file.html
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

Another bot bites the dust.