Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Does Googlebot always use its own user agent string?

         

graeme_p

9:45 am on Sep 1, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I did a search that brought up several pages that show information about the visitors browser.

The usual way that info shows in the snippet in the SERPS is something like this:

Your Browser User Agent String is Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html).


But one site shows:

Platform Linux x86_64 Agent Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0


How did that happen? It does not seem to be a fault in the site which accurately shows the UA for obscure browsers and platforms (so its must be just echoing the US the browser sends).

There is not something complex happening in the snippet generation either - it remains the same for everything from Firefox on Linux to Midori to Chrome on Android.

incrediBILL

10:35 am on Sep 1, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



First, do you do a reverse DNS of the IP address?

It will always show that it's googblebot in the return host name.

However, Google also makes screen previews, or they used to and may still be for some sites and that would explain the browser user agent although I thought they used Chrome but I can't remember off the top of my head.

lucy24

9:16 pm on Sep 1, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Google also makes screen previews

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML, like Gecko; Google Web Preview) Chrome/22.0.1229 Safari/537.4

It has to be a recent humanoid UA so the site serves up approximately what it would serve to a human.

graeme, it would help if you showed the IP. Watch out for 66.249; DomainTools lives right next door to google and will show up with the human user's UA.

graeme_p

7:50 am on Sep 2, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I do not know the IP - the UA showed up the the snippet for a site in the SERPS, and the IP did not. Its not my site!

The snippet for some other sites that show browser info shows Googlebot, this does not. It is also not the UA Lucy says is used form previews.

The Web Preview UA is just hideous - it contains the names of three rendering engines AND two browsers AND what it actually is.

lucy24

8:33 am on Sep 2, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The Web Preview UA is just hideous

:: detour to check something in raw logs ::

Humans:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.89 Safari/537.36
Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36
Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31

Those are just the first three hits from searching for "Linux.+Chrome". They're really no different from the Preview UA, apart from the parenthetical extra.

Chrome's UA always includes "Safari". Annoying, but there you are.

graeme_p

8:51 am on Sep 2, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yes, its just a slightly worse version of the Chrome UA - but that is pretty bad already. It includes the names of three rendering engines, none of which is accurate - Google uses a fork of Apple's Webkit, which is a fork of KHTML and completely unrelated to Gecko.