Welcome to WebmasterWorld Guest from 35.173.57.202

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

WMT Fetch as Googlebot

     
12:08 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


Hi,
I am confused.
I just read that google index javascript then I read that google doesn´t.
So I checked with google webmaster tool - explorer like google.
I thought that what you can see if you click on correct (code of page) is what google see and index.
I am doing a slideshow with images and text, and of course I want the text indexed if possible. So I went into google webmaster tool - explorer like google, and all text thats in the javascript comes up, and all other javascript also.

So does google index that text or not?
Thanks
12:45 pm on Nov 27, 2013 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Apr 30, 2008
posts:2630
votes: 191


I am presuming that by "explorer like google" you mean "Fetch as googlebot" option in WMT.

Once you did "Fetch as Googlebot", if a page returned 200 OK, then clicking on the code will show you what Google sees. It does not necessarily mean that Google will index that page.

If the page is indexed, Google will index the text on the page. The javascript itself will not be "indexed" as content of the page. But if there is a link within javascript, Google may follow that link.
1:14 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


Yes, I meant: "Fetch as googlebot",
I did a literary translation froms spanish WMT :)
ok, thanks,
so I can assume the text will not be indexed.
however by the text in the script, it looks like it should be indexed however if I view the page with javascript disabled I dont see anything between the script tags.
This is a part of the script where images and text are placed:

roller.add({
img : 'images/tab-layout.png',
text : '<h1>Tab layout</h1><p>Se example of the Tab layout model in ludoJS.</p>' +
'<p>The navigation tabs can be placed on top or at one of the other edges, left, right or bottom.</p>',
url : 'http://www.example.com/tab.html'
});

I am using, html. 4, has not had time yet to learn and change to html 5.

Is there a way to do a slideshow, with images and text that does not use javascript?
thanks
9:05 pm on Nov 27, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15802
votes: 845


Seems like you're asking about two different things: indexing javascript, and acting on javascript. Personally I take no chances, and have this in htaccess:

<FilesMatch "\.(js|txt|xml)$">
Header set X-Robots-Tag "noindex"
</FilesMatch>

Crawl, yes. Execute, yes. But do you want the actual content of your js files to show up in searches?

Do you have pages that are only accessible via javascript? Have the major search-engine robots crawled these pages?
10:11 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


Seems like you're asking about two different things: indexing javascript, and acting on javascript. Personally I take no chances, and have this in htaccess:

<FilesMatch "\.(js|txt|xml)$">
Header set X-Robots-Tag "noindex"
</FilesMatch>

Crawl, yes. Execute, yes. But do you want the actual content of your js files to show up in searches?

Do you have pages that are only accessible via javascript? Have the major search-engine robots crawled these pages?

Hi Lucy,
I am redesigning the site, trying to get an acceptable design ;) but without loosing the texts I had before.
What do you mean with the actual content of the js. file to show up in searches?

If I understand you correctly, no I dont want the code indexed, however the texts and links I want indexed, same as I had before.
I have found an slideshow in wich I can add the text and links in a div, and will be overwritten when javascript is enabled and slideshow run.
Can google see that as spam? The text will be identical as the one displayed inside the javascript.
As does not seem 100% google index it or not,
So if google index the .js, the text and link will be doubble so better restrict google to not index it.

Why do you set javascript to noindex?
Thanks
10:30 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:July 19, 2013
posts:1097
votes: 0


roller.add({
img : 'images/tab-layout.png',
text : '<h1>Tab layout</h1><p>Se example of the Tab layout model in ludoJS.</p>' +
'<p>The navigation tabs can be placed on top or at one of the other edges, left, right or bottom.</p>',
url : 'http://www.example.com/tab.html'
});

Below the </script> [or where you would like the images/text to appear for those not using JS] use a <noscipt> like described below.

<noscript>
<p>Put the contents [images,text,links,etc.] you want to make sure bots and those not executing JS for some reason see -- Then it won't matter if a visitors "runs the javascript" to see what you're presenting. They'll get the same info/images without JS, but won't have the use of the slideshow or any other JS dependent actions.</p>
</noscript>
10:42 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


<noscript>
<p>Put the contents [images,text,links,etc.] you want to make sure bots and those not executing JS for some reason see -- Then it won't matter if a visitors "runs the javascript" to see what you're presenting. They'll get the same info/images without JS, but won't have the use of the slideshow or any other JS dependent actions.</p>
</noscript>

Yes thanks, forgot about it, was just confused to see that it seemed like google indexed the javascript content.
So the best the good old way.
10:50 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:July 19, 2013
posts:1097
votes: 0


Yeah, and I doubt it's going to be a big deal <noscript> to much longer, especially with browser producers like Mozilla removing the ability to disable JS except via an extension since v23 of FireFox.

Search bots and screen readers are catching up to full browser functionality if they're not already there, so I doubt it's going to be an issue much longer since most can already run it, some can't even turn it off unless they work to turn it off and without it many, many, many sites don't work the way they should, but still, better safe than sorry until we know "we're there" and don't need to worry about it any more.
10:57 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


Yeah, and I doubt it's going to be a big deal <noscript> to much longer, especially with browser producers like Mozilla removing the ability to disable JS except via an extension since v23 of FireFox.

In firefox, if you write about:config in the browser you can disable javascript.
11:00 pm on Nov 27, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:July 19, 2013
posts:1097
votes: 0


Ah, cool, thanks!

Of course, for the average FF user (and even some of us who use it often but don't really care to turn JS off, except once every 5 or 6 months to check something) that's like having to have an extension, only different lol :)
11:25 pm on Nov 27, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15802
votes: 845


What do you mean with the actual content of the js. file to show up in searches?

That's assuming some of your scripts are in separate files. If someone did an exact-text search for, say, "The navigation tabs can be placed on top or at one of the other edges" they might find one of your pages-- but you wouldn't want them to find
:: detour to make sure this isn't the filename you really use ::
www.example.com/helenscripts.js
8:35 am on Nov 28, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


That's assuming some of your scripts are in separate files. If someone did an exact-text search for, say, "The navigation tabs can be placed on top or at one of the other edges" they might find one of your pages-- but you wouldn't want them to find
:: detour to make sure this isn't the filename you really use ::
www.example.com/helenscripts.js

Yes and no,
I have small scripts together not to have so many requests, but also other script separate such as motools.

However the example you gave: "The navigation tabs can be placed on top or at one of the other edges"
Sounds like a comment in the javascript file, and if google read and index javascript I suppose it should not read the comments.
8:07 pm on Nov 28, 2013 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11817
votes: 235


Yeah, and I doubt it's going to be a big deal <noscript> to much longer, especially with browser producers like Mozilla removing the ability to disable JS except via an extension since v23 of FireFox.

safari and opera have native options to disable javascript.
8:26 pm on Nov 28, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


safari and opera have native options to disable javascript.

Are there any figures wich % have javascript disabled?
10:08 pm on Nov 28, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15802
votes: 845


You can test it easily if you feel like doing the work.

<noscript>
<img src = "images/singlepix.gif" width = "1" height = "1" alt = "">
</noscript>

Shove into your universal footer, sit back for a week or so and look at requests for /singlepix.gif as a proportion of total human requests. If you're running analytics this information should already be available; I don't know about GA, but piwik has a no-script alternative that works on this principle.

Honestly I don't think there are many, though. People at WebmasterWorld have got to be overrepresented among both noscript and cookie-blocker types.
10:15 pm on Nov 28, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2003
posts: 1318
votes: 0


You can test it easily if you feel like doing the work.

<noscript>
<img src = "images/singlepix.gif" width = "1" height = "1" alt = "">
</noscript>

Shove into your universal footer, sit back for a week or so and look at requests for /singlepix.gif as a proportion of total human requests. If you're running analytics this information should already be available; I don't know about GA, but piwik has a no-script alternative that works on this principle.

Honestly I don't think there are many, though. People at WebmasterWorld have got to be overrepresented among both noscript and cookie-blocker types.

Smart idea, however not sure have time and interest enought to investigate.

Talking about cookies, not sure you know about the new cookie law in Eu, it says you have to advice that the site uses cookies, such as google analytics for exempel, and must bee easily seen, most use a banner.
However in spain they gone even far, telling one can't install any cookie until the surfer accepted to do so, thing that maybe 2% of website do, most just tell they use them and how to delete them.
Its so crazy, all sites full of banners, telling they use cookies. Its really awful surfing.
I guess nobody use noscript for those banners....
This is something to open a separate post.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members