Page is a not externally linkable
- Google
-- Google News Archive
---- "Googlebot/Test" Pulling External Javascipt?


Dolemite - 5:05 pm on Mar 18, 2004 (gmt 0)


It does obey robots.txt as my javascripts are in a disallowed directory and were not spidered.

So because a rarely-seen spider didn't pull your particular .js files, you assume it obeys robots.txt?


Thread source:: http://www.webmasterworld.com/google_archive/22740.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com