Forum Moderators: open
65.192.195.16 - - [date] "GET /robots.txt HTTP/1.0" 404 836 "-" "teoma_agent3 [crawler-admin@teoma.com]"
The *.16 seems to look for "robots.txt" files, while:
63.236.92.151 - - [08/Apr/2001:06:56:46 -0600] "GET /pagename.html HTTP/1.0" 200 695 "-" "teoma_agent3 crawler-admin@teoma.com"
63.236.92.* Seems looks for named pages.
[teoma.com...]
65.192.195.16 - - [04/Apr/2001:01:48:01 -0600]
65.192.195.16 - - [07/Apr/2001:22:37:30 -0600]
63.236.92.151 - - [08/Apr/2001:06:56:46 -0600]
63.236.92.147 - - [09/Apr/2001:09:08:49 -0600]
63.236.92.150 - - [09/Apr/2001:09:42:01 -0600]
63.236.92.149 - - [09/Apr/2001:10:11:32 -0600]
63.236.92.149 - - [09/Apr/2001:11:04:59 -0600]
63.236.92.150 - - [09/Apr/2001:11:08:03 -0600]
63.236.92.146 - - [09/Apr/2001:11:32:48 -0600]
63.236.92.149 - - [09/Apr/2001:11:44:10 -0600]
63.236.92.150 - - [09/Apr/2001:11:51:24 -0600]
63.236.92.152 - - [09/Apr/2001:12:09:25 -0600]
63.236.92.152 - - [09/Apr/2001:12:35:11 -0600]
63.236.92.147 - - [09/Apr/2001:15:32:43 -0600]
65.192.195.16 - - [11/Apr/2001:16:06:05 -0600]
65.192.195.16 - - [12/Apr/2001:18:21:08 -0600]
65.192.195.16 - - [13/Apr/2001:23:05:29 -0600]
63.236.92.150 - - [15/Apr/2001:02:49:00 -0600]
What do you think of this?