homepage Welcome to WebmasterWorld Guest from 23.22.173.58
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Recent ip's list
Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 603 posted 11:06 am on Apr 29, 2001 (gmt 0)

[xoc.net]

 

praxis

10+ Year Member



 
Msg#: 603 posted 3:49 pm on May 1, 2001 (gmt 0)

I have seen a lot of activity, as far back as Dec. so far, from the following:

65.192.195.16 - - [date] "GET /robots.txt HTTP/1.0" 404 836 "-" "teoma_agent3 [crawler-admin@teoma.com]"

The *.16 seems to look for "robots.txt" files, while:

63.236.92.151 - - [08/Apr/2001:06:56:46 -0600] "GET /pagename.html HTTP/1.0" 200 695 "-" "teoma_agent3 crawler-admin@teoma.com"

63.236.92.* Seems looks for named pages.

[teoma.com...]

65.192.195.16 - - [04/Apr/2001:01:48:01 -0600]
65.192.195.16 - - [07/Apr/2001:22:37:30 -0600]
63.236.92.151 - - [08/Apr/2001:06:56:46 -0600]
63.236.92.147 - - [09/Apr/2001:09:08:49 -0600]
63.236.92.150 - - [09/Apr/2001:09:42:01 -0600]
63.236.92.149 - - [09/Apr/2001:10:11:32 -0600]
63.236.92.149 - - [09/Apr/2001:11:04:59 -0600]
63.236.92.150 - - [09/Apr/2001:11:08:03 -0600]
63.236.92.146 - - [09/Apr/2001:11:32:48 -0600]
63.236.92.149 - - [09/Apr/2001:11:44:10 -0600]
63.236.92.150 - - [09/Apr/2001:11:51:24 -0600]
63.236.92.152 - - [09/Apr/2001:12:09:25 -0600]
63.236.92.152 - - [09/Apr/2001:12:35:11 -0600]
63.236.92.147 - - [09/Apr/2001:15:32:43 -0600]
65.192.195.16 - - [11/Apr/2001:16:06:05 -0600]
65.192.195.16 - - [12/Apr/2001:18:21:08 -0600]
65.192.195.16 - - [13/Apr/2001:23:05:29 -0600]
63.236.92.150 - - [15/Apr/2001:02:49:00 -0600]

What do you think of this?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved