Welcome to WebmasterWorld Guest from 18.205.246.238

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google WMT Now Includes a Blocked Resources Report

     
6:57 pm on Mar 11, 2015 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:25780
votes: 831


I'm pleased to see this addition to GWT.

That report will be useful as it may be an unknown before a third party indicates a problem.



Webmasters often use linked images, CSS, and JavaScript files in web pages to make them pretty and functional. If these resources are blocked from crawling, then Googlebot can't use them when it renders those pages for search. Google Webmaster Tools now includes a Blocked Resources Report to help you find and resolve these kinds of issues.Google WMT Now Includes a Blocked Resources Report [googlewebmastercentral.blogspot.com]
11:07 pm on Mar 11, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member netmeg is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Mar 30, 2005
posts:13007
votes: 222


This is going to be really useful for me.
11:41 pm on Mar 11, 2015 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:June 19, 2005
posts: 361
votes: 12


Will be useful if they update the data. Most of my problems listed have been fixed for over a month.
3:52 am on Mar 12, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15372
votes: 724


By the ordinary yawn-provoking coincidence, earlier today I discovered by accident that if google has the hiccups and fails to download the stylesheets belonging to a particular page, they will unequivocally declare that the page is NOT mobile-friendly. So presumably they will make the same claim if stylesheets are blocked in robots.txt.

:: detour to look ::

Incidentally, it's under the Google Index tab in wmt. You'd expect the Crawl tab, wouldn't you?

### ! I'd entirely forgotten that certain ebooks' images are blocked to the imagebot due to excessive hotlinking. But I'm ### if I see why I should let them crawl music files (midi, mp3) when there's no such index.

Click a host on the report to see a list of blocked resources from that host. Go through the list and start with those that might affect the layout in a meaningful way. Less important resources, such as tracking pixels or counters, aren't worth bothering with.

Uh... I know which resources affect page layout and which ones don't. But how will google know? What's missing is something analogous to the parameters tool, where you can tell them which resources affect layout.
8:01 am on Mar 12, 2015 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Sept 7, 2006
posts: 1047
votes: 99


Incidentally, it's under the Google Index tab in wmt. You'd expect the Crawl tab, wouldn't you?


Especially when JM's post says: "An update to Fetch and Render shows how these blocked resources matter" (which is another new feature that is under the Crawl tab). It took me some time to find it.

I'm not sure what it is supposed to rell me that I didn't already know. Presumably most of us know what is in our own robots.txt files, although I suppose anyone calling on resources that are hosted elsewhere might find it useful.
1:39 pm on Mar 12, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member bwnbwn is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 25, 2005
posts:3589
votes: 45


lucy24 funny you said
has the hiccups and fails to download the stylesheets belonging to a particular page
I checked a mobile website I had built about 3 months ago just for the heck of it to see if G recognized it as mobile. I have a desktop and a m.website.com version. The mobile version is detected by java script. When I tested it months ago G detected it fine and I tested it several times after to make sure. Yesterday I tested it and got no mobile so I retested it got the same. After 4 test G finally recognized it as mobile I wonder if there are more hiccups going on other than stylesheets.
2:19 pm on Mar 12, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3509
votes: 320


Lucy wrote:
I'd entirely forgotten that certain ebooks' images are blocked to the imagebot due to excessive hotlinking.

Lucy, you can use an x-robots instruction in .htaccess to let Google crawl your images, but not show them in public searches.
<Files ~ "\.(gif|jp[eg]|png)$">
Header append x-robots-tag "noindex"
</Files>
7:01 pm on Mar 12, 2015 (gmt 0)

Senior Member from AU 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 22, 2003
posts: 2226
votes: 139


GWT Message:
Hm. Something isn't right. We're checking into it now.
7:19 pm on Mar 12, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15372
votes: 724


you can use an x-robots instruction in .htaccess to let Google crawl your images, but not show them in public searches.

... and I know this perfectly well, because I've actually got this directive in other contexts, although I'd forgotten all about it. Oops. For me it should be absurdly simple, because in each case the blocked images live in a directory of their own, so I don't even need a <Files> or <FilesMatch> envelope.

That should work. I think the imagebot learns pretty quickly that repeat requests for images almost invariably get a 304, so there will be almost no change in crawling.
9:07 pm on Mar 12, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3509
votes: 320


Hm. Something isn't right. We're checking into it now.

Who is checking into it?
3:49 am on Mar 13, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


As usual the tool places the error on me instead of Google itself:
http://pagead2.googlesyndication.com/pagead/show_ads.js Script Blocked
http://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js Script Blocked
https://oauth.googleusercontent.com/gadgets/js/core:rpc:shindig.random:shindig.sha1.js?c=2 Script Blocked
https://ssl.gstatic.com/accounts/o/2006856415-postmessagerelay.js Script Blocked
Reminds me of the cop that gave me an illegal U-turn ticket, then made that same U-turn as he drove away.
3:54 am on Mar 13, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


you can use an x-robots instruction in .htaccess to let Google crawl your images, but not show them in public searches.

Works like a charm with Google. Too bad Yandex, Bing, Yahoo, Duck Duck et al don't support x-robots tag. They only support image files disallowed in robots.txt, thus the quagmire.
1:08 am on Mar 24, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 30, 2007
posts: 684
votes: 4


I am seeing this on my partner's website and the blocked scripts are the Google + button... (the pages it isn't on don't have the render problem) so how to resolve it? (apart from not having the number of Google pluses etc)

[ssl.gstatic.com...]

[oauth.googleusercontent.com...]
1:21 am on Mar 24, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


@Gemini23

IMO there isn't anything to resolve. Google's tool is just listing blocked resources across the board, even it's own. My pages render fine even though Google+ and other Google properties are blocked by Google itself. Several of these tools give similar ambiguous reports.
2:08 am on Mar 24, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member editorialguy is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:June 28, 2013
posts:3360
votes: 707


Google's tool is just listing blocked resources across the board, even it's own.


Yes, like embedded Google Maps.
4:08 am on Mar 24, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891



Yes, like embedded Google Maps.

I use Google maps all over my sites, but wrap them in JavaScript called externally. So far this method is not showing as blocked :)