I have 71 hits within 1 minute 27 seconds from: 188.8.131.52 - - [24/Jun/2001:15:26:29 -0700] "GET /****.html HTTP/1.0" 200 6220 "-" "Mozill
(edited by: mdharrold at 3:40 am (gmt) on June 27, 2001
4:40 am on Jun 27, 2001 (gmt 0)
184.108.40.206 resolves to the IP block used by capitalnet.com.cn, a Chinese ISP tagetted at business. It could be a dialup account, or a small company, or... As to why they are spidering you, depends on your site.
Hope this hbelps
8:50 pm on Jun 27, 2001 (gmt 0)
My copy and paste did not show up entirely above 220.127.116.11 - - [24/Jun/2001:15:25:11 -0700] "GET /*.html HTTP/1.0" 200 8884 "-" "Mozilla/3.0 (compatible; Indy Library)"
Thanks for the IP info Woz. If that is a dial up IP, can you run a spider on a dial up?
11:31 pm on Jun 27, 2001 (gmt 0)
In theory, yes. A robot is just a program running on a computer that can crawl webpages through http requests. It can run on a dial up from the computer it is on but that is not something any serious buisness would do :)
6:10 am on Jul 1, 2001 (gmt 0)
I also have been hit by Indy Library (but not recently) and find it a pain. You can find some more info on it in this thread: