Forum Moderators: martinibuster

Message Too Old, No Replies

Googlebot crawl bot exercising AdSense Javascript?

It's in my JavaScript-based AdSense tracking logs

         

RonS

7:13 pm on Jun 8, 2005 (gmt 0)

10+ Year Member



I have, on every page on my site that contains AdSense, some javacode (that I picked up here at WW, thank you very much). This javascript tracks most of my AdSense clicks very nicely.

I've been running the software for a few months.

Every few days or maybe once a week, but fairly infrequently, I will find a couple of reports of my script having been activated by crawl.googlebot.

It's interesting to not that I don't see any referral/page/time info on these activations, as I do on every other activation.

What I don't know is if the googlebot is activating my javascript or my scriptfile directly thinking it's an image of some sort, or if it's google activating it's AdSense javascript, which in turn is activating my script.

I think it's likely that it is the googlebot directly accessing the scriptfile on my site, since none of the other variables are in place... I don't know what would happen if a bot activated the adsesne or the tracking javascript. I should note that in looking through G's index of some of my smaller sites, I have seen the URL of the tracking code scriptfile in the site:<domain> results.

Here's the point of all of this: I play it totally clean, so I'm not worried about getting a boot, and if it's G's intent to monitor the pages that contain AdSense that's absolutely fine by me. The point is that, at a minimum, Googlebot is reading the javascript and following links in it.

This is news to me. I have always read that if you want to "hide links" and do other sneaky stuff that the way to go was to embed things in Javascipt since the 'bots don't look at it, though I always that was silly, since I'm sure that G has some smart people who could figure out how to exercise javascipt in a bot.

So what say you all? Have you seen these googlebot activations? Is G checking on our use of tracking systems? Do you hide links in JavaScript that have been "outted" by G?

Thanks.

RonS

6:43 am on Jun 9, 2005 (gmt 0)

10+ Year Member



Nobody has ANY thoughts on this at all?

Wow. Slow day I guess.

Gandhalf

7:17 am on Jun 9, 2005 (gmt 0)

10+ Year Member



I do.
I experienced the exact same thing.
It still seems to be unclear wether Google reads and follow Javascript links.

kokaroach

7:20 am on Jun 9, 2005 (gmt 0)

10+ Year Member



Try going to Google and typing in "keyword" filetype:js

Google reads just about everything out there on the web these days. You can also try filetype:xml, filetype:pdf, filetype:cgi, etc.

If it's out there, G can probably crawl it and index it.

K

Jenstar

1:48 pm on Jun 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Does the IP resolve to Google? If so, it could be someone within Google quality checking with the AdSense UA, to check for cloaking.

If it doesn't resolve to Google, it could be someone surfing with the same UA for any number of reasons.

ve3cnu

5:08 pm on Jun 9, 2005 (gmt 0)

10+ Year Member



I read elsewhere on a professional bloggers blog that Google's new web accelerator beta was doing this.

RonS

9:47 pm on Jun 9, 2005 (gmt 0)

10+ Year Member



Yes, Jenstar, the IPs do infact resolve to crawl.google#*$!xxxxx through rDNS according to whois.source (no link ;) )

It's not a normal browser that's doing this, based on the behavior and reporting of the script, and the ONLY time I see this behavior (or anything similar) it is associated with a google IP.

Yes, my sites are spidered by many many SEs.

I suppose it could be the prefetch toolbar, but would the IP be associated with google and not the browser? I doubt it, but anything is possible I suppose.

Rodney

9:50 pm on Jun 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I suppose it could be the prefetch toolbar, but would the IP be associated with google and not the browser?

I think that's how the prefetch work. It sends all traffic through the google servers first (google servers prefetch the links on the site).