Page is a not externally linkable
- Google
-- Google News Archive
---- "Googlebot/Test" Pulling External Javascipt?


Yidaki - 6:15 pm on Mar 20, 2004 (gmt 0)


>Google has been spidering and indexing my javascript files for maybe a year now.

Forgive me but i simply don't believe you. I haven't seen this on any of the 50+ sites i manage for me and my clients - many of these sites use javascript for many things - from popup to document write. Never ever had any single js crawled by gbot nor any other major bot.

Enough reasons for me to say that you're either wrong or simply just trying to blow another bubble.

Regarding GBot/Test ignoring robots.txt: it wouldn't surprise me. Doh! A bot that doesn't identify itself with a user agent that contains correct contact info and/or correct whois info +plus+ GoogleGuy claiming that he doesn't know anything about a Testbot ... sorry, but this stinks ...


Thread source:: http://www.webmasterworld.com/google_archive/22740.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com