Welcome to WebmasterWorld Guest from

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Is Google now using Chrome to crawl the web ?

5:11 am on Oct 7, 2008 (gmt 0)

Preferred Member

joined:Dec 28, 2005
votes: 0

Haven't posted in a long time. Thought i would drop in to report a very interesting Google activity I've been observing today.

I have an "who's online" script running and reporting visitors in real time as they browse the site. The script uses javascript to report referrer and currently viewed page.

One of the visitors today was from [google.com] sucking approx 200 pages in a 15-20 minutes frame and invoking the on page tracking javascript....sending referring page and current URL information (it had to invoke javascript just like a browser would to do that). In other words acting very much like it is a full on browser.

Maybe it is already old news to some, maybe i missed previous topics disscussing this. Anyway, it is the first time i see this thing in real time. Very interesting crawl development.

Anyone else noticed this ?

Page to page browsing was occuring at a very fast rate, could not be human.

7:11 pm on Oct 20, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 8, 2003
votes: 0

jecasc - What identified this as a Google crawler?

The "who is online" shows the IP. I checked the IP on an IP lookup website and the IP was from Google. Also the requests came so fast that it could not possibly be a human search. Perhaps I would not have even noticed it if the search terms would not have been so odd. I also checked if there was a website somewhere linking to my website with the search terms, but I found none.

The whois online is not javascript based however like Web_speed's. (It is an oscommerce webshop.)

I have checked my logfiles of the past days and I found the same happening a few days ago:

User Agent: "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

This 31 message thread spans 2 pages: 31