Forum Moderators: DixonJones

Message Too Old, No Replies

Googlebot in user agents

Fake or does Google do this

         

frugalsnow

8:06 pm on Apr 11, 2005 (gmt 0)

10+ Year Member



I'm interested in understanding how an otherwise normal user-agent gets "Googlebot" appended to it.

Here are some user agents from a recent web server log. All of these agents contain "Googlebot", but most of them do not come from a Google IP address, as far as I can tell. There were several requests with each of these agents over the course of 2 weeks.

Does Google have a browser, toolbar or something that would insert "Googlebot" into a regular user's user-agent? I think the pattern of these makes it unlikely that different users are inserting "Googlebot" into their otherwise normal user agent strings.

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705; Googlebot)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; Googlebot/2.1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; Googlebot; .NET CLR 1.0.3705; Googlebot)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; Googlebot; .NET CLR 1.0.3705; Googlebot)0
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322; Googlebot)
VM4050/132.037 UP.Browser/6.2.2.4.e.1.100 (GUI) MMP/2.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (compatible; Googlebot/2.1; Googlebot/2.1; .NET CLR 2.0.40607; .NET CLR 1.0.3705)

bhartzer

8:14 pm on Apr 11, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I would be that the user-agens it being faked. Googlebot usually uses a Google IP--although I've heard that they will sometimes not use anything that gives themselves away if they're checking for cloaking.

Personally, if I saw that in any of my logs I would block it.

MatthewHSE

8:27 pm on Apr 11, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've heard of folks spoofing Googlebot to gain access to member-only sections of some news sites, etc. These people may have changed their UA string for that purpose.