Google WMT Now Includes a Blocked Resources Report

[engine]

I'm pleased to see this addition to GWT.

That report will be useful as it may be an unknown before a third party indicates a problem.

Webmasters often use linked images, CSS, and JavaScript files in web pages to make them pretty and functional. If these resources are blocked from crawling, then Googlebot can't use them when it renders those pages for search. Google Webmaster Tools now includes a Blocked Resources Report to help you find and resolve these kinds of issues.Google WMT Now Includes a Blocked Resources Report [googlewebmastercentral.blogspot.com]

[netmeg]

This is going to be really useful for me.

[dethfire]

Will be useful if they update the data. Most of my problems listed have been fixed for over a month.

[lucy24]

By the ordinary yawn-provoking coincidence, earlier today I discovered by accident that if google has the hiccups and fails to download the stylesheets belonging to a particular page, they will unequivocally declare that the page is NOT mobile-friendly. So presumably they will make the same claim if stylesheets are blocked in robots.txt.

:: detour to look ::

Incidentally, it's under the Google Index tab in wmt. You'd expect the Crawl tab, wouldn't you?

### ! I'd entirely forgotten that certain ebooks' images are blocked to the imagebot due to excessive hotlinking. But I'm ### if I see why I should let them crawl music files (midi, mp3) when there's no such index.

Click a host on the report to see a list of blocked resources from that host. Go through the list and start with those that might affect the layout in a meaningful way. Less important resources, such as tracking pixels or counters, aren't worth bothering with.

Uh... I know which resources affect page layout and which ones don't. But how will google know? What's missing is something analogous to the parameters tool, where you can tell them which resources affect layout.

[Wilburforce]

Incidentally, it's under the Google Index tab in wmt. You'd expect the Crawl tab, wouldn't you?

Especially when JM's post says: "An update to Fetch and Render shows how these blocked resources matter" (which is another new feature that is under the Crawl tab). It took me some time to find it.

I'm not sure what it is supposed to rell me that I didn't already know. Presumably most of us know what is in our own robots.txt files, although I suppose anyone calling on resources that are hosted elsewhere might find it useful.

[bwnbwn]

lucy24 funny you said

has the hiccups and fails to download the stylesheets belonging to a particular page

I checked a mobile website I had built about 3 months ago just for the heck of it to see if G recognized it as mobile. I have a desktop and a m.website.com version. The mobile version is detected by java script. When I tested it months ago G detected it fine and I tested it several times after to make sure. Yesterday I tested it and got no mobile so I retested it got the same. After 4 test G finally recognized it as mobile I wonder if there are more hiccups going on other than stylesheets.

[aristotle]

Lucy wrote:
I'd entirely forgotten that certain ebooks' images are blocked to the imagebot due to excessive hotlinking.

Lucy, you can use an x-robots instruction in .htaccess to let Google crawl your images, but not show them in public searches.

<Files ~ "\.(gif|jp[eg]|png)$">
Header append x-robots-tag "noindex"
</Files>

[IanCP]

GWT Message:

Hm. Something isn't right. We're checking into it now.

[lucy24]

you can use an x-robots instruction in .htaccess to let Google crawl your images, but not show them in public searches.

... and I know this perfectly well, because I've actually got this directive in other contexts, although I'd forgotten all about it. Oops. For me it should be absurdly simple, because in each case the blocked images live in a directory of their own, so I don't even need a <Files> or <FilesMatch> envelope.

That should work. I think the imagebot learns pretty quickly that repeat requests for images almost invariably get a 304, so there will be almost no change in crawling.

[aristotle]

Hm. Something isn't right. We're checking into it now.

Who is checking into it?

[keyplyr]

As usual the tool places the error on me instead of Google itself:

http://pagead2.googlesyndication.com/pagead/show_ads.js Script Blocked
http://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js Script Blocked
https://oauth.googleusercontent.com/gadgets/js/core:rpc:shindig.random:shindig.sha1.js?c=2 Script Blocked
https://ssl.gstatic.com/accounts/o/2006856415-postmessagerelay.js Script Blocked

Reminds me of the cop that gave me an illegal U-turn ticket, then made that same U-turn as he drove away.

[keyplyr]

you can use an x-robots instruction in .htaccess to let Google crawl your images, but not show them in public searches.

Works like a charm with Google. Too bad Yandex, Bing, Yahoo, Duck Duck et al don't support x-robots tag. They only support image files disallowed in robots.txt, thus the quagmire.

[Gemini23]

I am seeing this on my partner's website and the blocked scripts are the Google + button... (the pages it isn't on don't have the render problem) so how to resolve it? (apart from not having the number of Google pluses etc)

[ssl.gstatic.com...]

[oauth.googleusercontent.com...]

[keyplyr]

@Gemini23

IMO there isn't anything to resolve. Google's tool is just listing blocked resources across the board, even it's own. My pages render fine even though Google+ and other Google properties are blocked by Google itself. Several of these tools give similar ambiguous reports.

[EditorialGuy]

Google's tool is just listing blocked resources across the board, even it's own.

Yes, like embedded Google Maps.

[keyplyr]

Yes, like embedded Google Maps.

I use Google maps all over my sites, but wrap them in JavaScript called externally. So far this method is not showing as blocked :)