Welcome to WebmasterWorld Guest from 54.196.214.35

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

New Googlebot

Nokia6820/2.0

     
3:02 am on Mar 2, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 21, 2005
posts:379
votes: 0


Full UA: Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

coming from 66.249.65.129

I've never seen this in our sites before. I guess it is pretending to be a WAP phone.

I'm wondering why they need a different bot for this? What does it see differently to the normal googlebot?

10:14 pm on Mar 2, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


Just saw it here, too, coming from 66.249.66.1
8:27 am on Mar 3, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts:2038
votes: 1


I've been wary of that bot and other Google mobiles/WAP phones (Wireless Application Protocol) since early last month when one of its ilk didn't ask for robots.txt and then proceeded to go every which way, hitting my bot traps along the way.

'Twas this UA, from 64.233.166.136, almost like the one you saw:

Nokia3510i/1.0 (05.30) Profile/MIDP-1.0 Configuration/CLDC-1.0 (Google WAP Proxy/1.0)

(Saw that last Nov., too, again from 64.233.166.136; again no robots.txt.)

FWIW...

I'm also not wild about the following info excerpt, emphasis mine, but have yet to specifically opt-out, preferring instead to rewrite G's WAP-related UAs* until the jury's back on whether they routinely ask for, let alone heed robots.txt.

-----
Source: Google Information for Webmasters
Need to remove content from Google's index? [google.com]

[...]

Remove transcoded pages [google.com]

Google Web Search on mobile phones allows users to search all the content in the Google index for desktop web browsers. Because this content isn't written specifically for mobile phones and devices and thus might not display properly, Google automatically translates (or "transcodes") these pages by analyzing the original HTML code and converting it to a mobile-ready format. To ensure that the highest quality and most useable web page is displayed on your mobile phone or device, Google may resize, adjust, or convert images, text formatting and/or certain aspects of web page functionality.

To prevent your web page(s) from being transcoded, please send a removal request to mobile-support@google.com.
-----

Thoughts? About the WAPs? About the transcoding? (The latter makes G's caching look like a walk in the park, copyright-wise.)

.
*Google WAPs

MOT-V171 UP.Browser/6.2.2.7 (GUI) MMP/1.0 UP.Link/6.3.0.0.0 (Google WAP Proxy/1.0)

Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

Nokia3510i/1.0 (05.30) Profile/MIDP-1.0 Configuration/CLDC-1.0 (Google WAP Proxy/1.0)

Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 (compatible; Googlebot/2.1; [google.com...]

2:11 pm on Mar 3, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:June 25, 2005
posts:179
votes: 1


[webmasterworld.com ]

pfui: "Google WAP Proxy/1.0" is a proxy, not a googlebot.

5:29 pm on Mar 3, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


Right. I do not believe this bot is using WAP proxy IP addresses.
7:20 am on Mar 4, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts:2038
votes: 1


Here's my, um, 'rogue robot' philosophy...

"If it looks like a bot, and acts like a bot, and hits bot traps -- it's banned."

: )

2:08 am on Mar 20, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 21, 2005
posts:379
votes: 0


I found this information, which confirms it is definitely a bot, similar to Googlebot-Image:

If you go to the newly added robots.txt tab in Google Sitemaps, at the very bottom there is a drop down list. Look inside that, and you'll see this:

Googlebot-Mobile : crawls pages for our mobile index

So I guess it can be banned via robots.txt if required. Presumably (?) it will request and obey it.

5:01 am on Mar 20, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts:2038
votes: 1


From robots.txt:

User-agent: Googlebot-Mobile
Disallow: /

Alas, didn't matter:

crawl-66-249-66-133.googlebot.com
[27/Feb/2006:19:25:25 -0800] "GET /robots.txt
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

crawl-66-249-66-133.googlebot.com
[27/Feb/2006:19:25:26 -0800] "GET /
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

crawl-66-249-66-133.googlebot.com
[28/Feb/2006:01:47:36 -0800] "GET /welcome.html
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

crawl-66-249-66-133.googlebot.com
[28/Feb/2006:01:48:45 -0800] "GET /dir/file.html
"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

Another bot bites the dust.