Welcome to WebmasterWorld Guest from 220.127.116.11 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Pubcon Platinum Sponsor 2014
Vspider designfreak msg:402515 9:18 am on Apr 22, 2001 (gmt 0) Had this spider visit me yesterday - only drew one file - any ideas:
littleman msg:402516 9:28 am on Apr 22, 2001 (gmt 0)
Got an IP? Son_House msg:402517 3:15 pm on Apr 24, 2001 (gmt 0)
Maybe this is the one designfreak is talking about? Found this in our logs yesterday. Made two requests.
18.104.22.168 - - [23/Apr/2001:19:41:28 -0400] "GET /robots.txt HTTP/1.0" 200 1046 "-" "vspider"
littleman msg:402518 5:54 pm on Apr 24, 2001 (gmt 0)
Well these guys are wide open right now. Have a look: [ ...] 22.214.171.124 Apparently it is a canned spider/se targeted at business micro portals
From -> [
"Verity was one of the first companies to recognize the value of-and deliver-full-text search solutions for intranet, e-commerce and business portals. Our achievements come from our patented technology, long-term corporate viability, financial strength, support of industry leaders and proven success across a rich, broad customer base....blah,blah,blah..."
Son_House msg:402519 9:03 am on May 16, 2001 (gmt 0)
Same ua but different ip. Made 2 requests one of which was for robots.txt.
126.96.36.199 - "vspider"
Defense Technical Information Center
8725 John J. Kingman Road Suite 0944 Fort Belvoir, VA 22060-6218 US
Looks like this is their site: [