homepage Welcome to WebmasterWorld Guest from 23.22.179.210
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Featured Home Page Discussion

    
Rendering pages with Fetch as Google
phranque




msg:4675121
 11:03 pm on May 27, 2014 (gmt 0)

google announced today on the webmaster central blog that the Fetch as Google feature of Webmaster Tools will now show how Googlebot would render the page.

http://googlewebmastercentral.blogspot.com/2014/05/rendering-pages-with-fetch-as-google.html [googlewebmastercentral.blogspot.com]:
In order to render the page, Googlebot will try to find all the external files involved, and fetch them as well. Those files frequently include images, CSS and JavaScript files, as well as other files that might be indirectly embedded through the CSS or JavaScript. These are then used to render a preview image that shows Googlebot's view of the page.


since this feature respects robots.txt, you will see the effects of any images, javascript and css that are excluded from crawling, such as broken navigation.

 

goodroi




msg:4675436
 3:39 pm on May 28, 2014 (gmt 0)

About time Google made it easier for me to test my spammy cloaked pages :)

keyplyr




msg:4675494
 8:28 pm on May 28, 2014 (gmt 0)

When it comes to Google's so called *rendering* tools, my dilemma has always been them not including the images I disallow in robots.txt. But if I allow these image directories, Google will include my image files in their Image-Search, thus inviting every hobby page and summer-vacation forum on the www to hot-link.

Browsers (whether desk-top or mobile) have no issue, only Google's rendering tools... so what's the point?

EditorialGuy




msg:4675522
 10:14 pm on May 28, 2014 (gmt 0)

What's the point? To let you see what Googlebot "sees." Some people will find the tool useful or reassuring; others will ignore it.

goodroi




msg:4675524
 10:39 pm on May 28, 2014 (gmt 0)

A free tool is better than no tool. I can see this being helpful for educating clients that don't understand best practices or how crawling works.

As much as I joke about Google, I don't forget that Google needs to address a wide range of webmasters from ignorant and/or stupid to web geniuses. This tool might not be for everyone but I do think it can be helpful for many.

motorhaven




msg:4675525
 10:41 pm on May 28, 2014 (gmt 0)

You can allow Google a fetch of images yet still keep it out of the index. Allow it in robots.txt, then send this header with the image:

Header set X-Robots-Tag "noindex"

keyplyr




msg:4675539
 12:25 am on May 29, 2014 (gmt 0)


You can allow Google a fetch of images yet still keep it out of the index. Allow it in robots.txt, then send this header with the image:

Header set X-Robots-Tag "noindex"


Thanks for the suggestion, but no... I've used the X-Robots header tag for years and still my images were scraped by both Bing and Google because at one point I didn't have the image directories disallowed in robots.txt. I have spent endless hours writing emails to them both, finally getting my images (10s of thousands) out of their image-search.

I'm not about to experiment again with this just so Google's little tool looks better.

derfmann




msg:4676189
 9:15 pm on May 30, 2014 (gmt 0)

big news for SEOs ... its now really easy to show customers how bad the website renders for google crawler, you can even check mobile device renderings.

@keyplyr why not prevent hotlinking with referer-check?

keyplyr




msg:4676239
 12:57 am on May 31, 2014 (gmt 0)


@derfmann - I do that and more.

keyplyr




msg:4676320
 10:31 am on May 31, 2014 (gmt 0)

Update on my above statement concerning image file indexing: I've just been in a discussion at Google Groups concerning X-Robots tags. I've been assured by a Google employee they now fully support this tag (not so years ago.)

phranque




msg:4676322
 11:10 am on May 31, 2014 (gmt 0)

thanks for following up on that, keyplyr.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved