ken_b
msg:4505973 | 3:42 pm on Oct 9, 2012 (gmt 0) |
I'm also interested in the answer to this question.
|
lucy24
msg:4506139 | 8:35 pm on Oct 9, 2012 (gmt 0) |
| should I block the javascript in my robots.txt file or shouldn't I block it ? |
|
Wrong question. The question is whether you should block it absolutely-- in config or htaccess-- because Preview, Translate and so on aren't bound by robots.txt.
Are you asking whether google can find the file, or whether they can read it? They can definitely find it; quick detour to raw logs confirms that they occasionally ask for .js files. Presumably for indexing purposes, which means it's time to whip out the
:: shuffling papers ::
<FilesMatch "\.(js|txt|xml)$">
Header set X-Robots-Tag "noindex"
</FilesMatch>
|
ken_b
msg:4507025 | 6:58 pm on Oct 11, 2012 (gmt 0) |
<FilesMatch "\.(js|txt|xml)$">
Header set X-Robots-Tag "noindex"
</FilesMatch> |
|
This goes in the htaccess file, right?
First, last, or?
|
lucy24
msg:4507138 | 12:31 am on Oct 12, 2012 (gmt 0) |
You mean, where in htaccess does it go? Doesn't matter. Ordering in htaccess only makes a difference within individual modules. This one's a free-standing directive. Unless you've got something else in htaccess that says "yes! index all you like! override all other instructions!" and it overlaps the js/txt/xml group. This does not seem likely.
They probably use js when constructing Previews. But I can't personally check, because my js tend to involve fonts-- or piwik tracking-- so the default version is the same as the <noscript> version.
|
ken_b
msg:4507152 | 2:01 am on Oct 12, 2012 (gmt 0) |
| You mean, where in htaccess does it go? |
|
Yes, thanks for the explanation.
Try as I might, I have a hard time getting a grip on this stuff.
|
|
