Welcome to WebmasterWorld Guest from 18.104.22.168
joined:Sept 12, 2012
joined:Dec 27, 2006
What reasons would a webmaster have to block such files from Google?
do you really think you can block spying with robots.txt
Oh, come on. Google's mind may not work like yours and mine, but they know a stylesheet when they see one.
It says that fonts.google and two of Google's Adsense .js files were blocked by robots.txt
Whose robots.txt blocked it?
You might want to delete your robots.txt file, too, if it's only a few lines long.
Header set X-Robots-Tag "noindex"
Now, iPhoned makes money from ads. It doesn't have a ridiculous amount of them, but because it uses an ad network a fair amount of scripts and pixels get loaded. My hypothesis was: if Google is unable to render the CSS and JS, it can't determine where the ads on your page are. In iPhoned's case, it couldn't render the CSS and JS because they were accidentally blocked in their robots.txt after a server migration.
We recommend making sure Googlebot can access any embedded resource that meaningfully contributes to your site's visible content or its layout
joined:Feb 25, 2011
[edited by: aakk9999 at 10:42 pm (utc) on Jun 21, 2014]
[edit reason] Unlinked sample URLs [/edit]