Alexa Crawls .js Files

Alexa crawled my .js files and fetched the URL’s in them

[jim_w]

Did anyone else notice that Alexa reads javascript files and then fetches the pages called by them?

crawl30-public.alexa.com - - [17/Apr/2003:03:19:26 -0500] "GET /robots.txt HTTP/1.0" 200 750 "-" "ia_archiver"
crawl30-public.alexa.com - - [17/Apr/2003:03:19:27 -0500] "GET /gbvar.js HTTP/1.0" 200 1808 "-" "ia_archiver"
crawl11-public.alexa.com - - [17/Apr/2003:03:34:33 -0500] "GET /pur_popup.html HTTP/1.0" 200 1730 "-" "ia_archiver"

pur_popup.html is a popup window I have with information on it called in the javascript file.

Do any other search engines do this?

[jeremy goodrich]

Not that I'm aware of, I've never seen a spider in my logs grabbing a javscript file.

[jim_w]

Guess it is time to update robots.txt with files called via javascript, add metas, et. al. (gr-rrrrrrrrrr)