lucy24 - 2:47 am on Mar 28, 2012 (gmt 0)
But if GoogleBot can't look at this file to check it contains "no evil", why should they trust your page/site? If they can't trust it, why should they rank it?
Because the contents of the file are none of their ### business, that's why. You might just as well say that if an /images directory is roboted-out, then google has no choice but to disregard any and all html pages that use those images. After all, you don't know what it might be a picture of-- or what else might be lurking in the directory.
var piwikTracker = Piwik.getTracker("http://www.example.com/piwik/piwik.php", 1);
I can't get into this folder! There must be something bad going on! Obviously all that "getTracker" business is just a smokescreen for some nefarious hanky-panky, cleverly concealed behind the tracker's default wording.
What to do, what to do?
I know! (Still speaking as the googlebot here.) I'll ask my pal Preview; he goes everywhere.
What the ###? Preview reports getting a consistent 403 slammed in his face. Is nothing sacred?
I wonder if they follow their own suggestions
I am struck by the number of "Allow" lines. What proportion of robots know what this means?