Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Matt Cutts asks webmasters: let googlebot crawl js and css


gethan - 4:36 am on Mar 28, 2012 (gmt 0)


Hey Matt - I'll unblock my js when google stops hotlinking my images on image search? deal?


The reason I disallow robots from js directories as the spiders execute various scripts which are intended only for real visitors - and starts running off following links to do with social interactions... causing ridiculous load.

The reason other sites do it might be for black hat seo purposes - but then wouldn't they just serve google one js and visitors another...


Thread source:: http://www.webmasterworld.com/google/4433745.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com