Welcome to WebmasterWorld Guest from 54.90.119.59

Forum Moderators: open

Message Too Old, No Replies

Googlebot showing as Browser

not as a Spider

     
5:26 pm on Jun 17, 2004 (gmt 0)

New User

10+ Year Member

joined:July 1, 2003
posts:32
votes: 0


I have read many messages from people stating that the GoogleBot has not visited in months. I was thinking the same thing. BUT...I just ran my webstats for last month using Webtrends Log Analyzer and noticed in my BROWSERS area (where IE, Netscape, etc shows up), I see GoogleBot.

It does not show up under Spiders....

Any idea why? Does this mean GoogleBot is visiting?

10:17 pm on June 17, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


They changed the UA string that is left in the raw log files, a month or two back, I seem to recall.

I'm not sure if that was all versions of Googlebot, or just some sort of limited test run.

10:41 pm on June 17, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


There's also a "fake" Googlebot visiting from 200-series IP addresses. The complete User-agent string of this bot is "Googlebot" -- That's it, nothing else. As opposed to the genuine Googlebot, whose UA string is currently
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

This fake user-agent may be what's following your stats program.

Jim

10:43 pm on June 17, 2004 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12172
votes: 61


Googlebot has always shown up in the Browser Statistics in WebTrends server edition. I've been using it for years as a secondary statistics program and have always seen Googlebot in that area, never in the spiders section.
11:59 pm on June 17, 2004 (gmt 0)

New User

10+ Year Member

joined:July 1, 2003
posts:32
votes: 0


The string is Googlebot/2.1 (+http://www.googlebot.com/bot.html) so I believe it is the true Googlebot.

As far as WebTrends always reporting it in Browsers, in the last 1.5 years I've been using this program and tracking this site, it's the first time I've seen it in Browsers.

Anyways...very interesting.

12:08 am on June 18, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 2, 2003
posts:724
votes: 0


I have always seen Googlebot, and lately msnbot in the browser statistics (using Webtrends Professional Suite 5.5) Not sure about the newer Webtrends packages.

Googlebot/2.1 ( [googlebot.com...]
msnbot/0.11 ( [search.msn.com...]

All the rest drop in to the Visiting Spiders section.

12:14 am on June 18, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:June 10, 2003
posts:410
votes: 0


I have recently seen an apparently authentic spider with referrer "GoogleBot/2.1" -- the IP address on the WHOIS record is listed as being theirs (although perhaps the IP is being spoofed, but if so, someone really is trying hard!). I am still getting infrequent visits from the "regular" Googlebot.

Note the the former is "GoogleBot" and the latter is "Googlebot"; the upper-case "B" made my web log software think it was a different spider .. which it probably is.

12:31 am on June 18, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Sept 29, 2000
posts:12095
votes: 0


A while back there was supposed to be a bot turned loose by Google coming off a different IP than the usual whose mission was to detect duplicate content. If I'm not mistaken a timed-release penalty was supposed to be imposed with a subsequent visit, and if the infraction was remedied by the return visit the site would be in the clear. FWIW that's how I understood it, so you might want to check for any possible discernible pattern for other than their usual range of IPs.

Some folks around have been suspecting and speculating lately that Google is going after duplicate content, so it might be worth some consideration, just in case.

7:22 pm on June 18, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Another test googlebot poked around in peoples CSS or javascript files a month or two back I seem to recall too.
8:27 pm on June 18, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 15, 2003
posts:1418
votes: 0


Ever since I put that Firefox tool on that lets me switch my User Agent, I've been browsing using the correct Googlebot UserAgent almost everywhere.

You would be amazed at the stuff you have access to ... many content-driven sites that normally require a login are wide open to the googlebot user agent.

Down side: lots of extra links :)