I'm getting a ton of hits from these two IPs. The user-agent string for the first IP is "Mozilla/4.0 (compatible ; MSIE 6.0; Windows NT 5.1)". For the second, "Mozilla/4.0 (compatible;)".
Each appears to be a valid browser. ie. not readily identified as a bot or spider.
Anybody have any insight on these IPs? Are they spiders / bots?
Thanks.
volatilegx
6:03 pm on Feb 19, 2005 (gmt 0)
One IP is a Roadrunner account. The other belongs to:
OrgName: General Dynamics Armament Systems Corporation OrgID: GDASC
My guess is that they are not search engine spiders.
hairy armpit
6:30 pm on Feb 19, 2005 (gmt 0)
Thanks for your thoughts.
Strangely, the 166 IP hit the SAME page 25 times each day, in roughly 56 minute intervals. The 65 IP hit a broad spectrum of pages on 3 different days, but often many within seconds, ie. it's not a human.
Any ideas?
Thanks.
volatilegx
2:30 pm on Feb 21, 2005 (gmt 0)
I'm sure you're right that both are robots, but spidering robots exist for many reasons beyond indexing for search engines. There are bots that look for copyright/trademark violations, email address scrapers, data miners, hackers looking for vulnerabilities... the list goes on and on.