lucy24 - 7:28 pm on May 1, 2012 (gmt 0)
what would they be able to do with/learn from our other analytics programs that run with js scripts?
In the case of piwik it isn't the js itself; that part is just a bit of code that in turn points to the meat of the analytics at piwik.php. But if I let them into the js they will follow its link to php (this is direct observation, not a guess) and frankly I don't want to speculate what information google would be able to extract from analytics. Maybe the googlebot just gets logged as a spurious visitor, but maybe they run wild and learn all kinds of things that a human browser wouldn't be able to learn.
Both the js and the php live in the /piwik/ directory, which is roboted-out. Generally the googlebot by that name follows robots.txt, but in this case for some reason they don't. Are they making some arcane exception for linked files as opposed to primary requests? Or are they simply snooping?
Besides: executing the .js may be on their nickel, but downloading it is on mine, at 20K a pop.