| 4:40 am on Jun 27, 2001 (gmt 0)|
18.104.22.168 resolves to the IP block used by capitalnet.com.cn, a Chinese ISP tagetted at business. It could be a dialup account, or a small company, or... As to why they are spidering you, depends on your site.
Hope this hbelps
| 8:50 pm on Jun 27, 2001 (gmt 0)|
My copy and paste did not show up entirely above 22.214.171.124 - - [24/Jun/2001:15:25:11 -0700] "GET /*.html HTTP/1.0" 200 8884 "-" "Mozilla/3.0 (compatible; Indy Library)"
Thanks for the IP info Woz.
If that is a dial up IP, can you run a spider on a dial up?
| 11:31 pm on Jun 27, 2001 (gmt 0)|
In theory, yes. A robot is just a program running on a computer that can crawl webpages through http requests. It can run on a dial up from the computer it is on but that is not something any serious buisness would do :)
| 6:10 am on Jul 1, 2001 (gmt 0)|
I also have been hit by Indy Library (but not recently) and find it a pain. You can find some more info on it in this thread:
| 6:39 am on Jul 1, 2001 (gmt 0)|
After feeding them blank pages for about a month they decided to leave me alone.
| 9:56 am on Jul 1, 2001 (gmt 0)|
| 1:57 pm on Jul 11, 2001 (gmt 0)|