Does anyone know of where I can get a list of known spiders?

Forum Moderators: open

Message Too Old, No Replies

10:26 pm on Aug 27, 2001 (gmt 0)

Is there a list of known spiders and their masters available anywhere?

8:09 am on Aug 28, 2001 (gmt 0)

Database of Web Robots [robotstxt.org]

10:42 am on Aug 28, 2001 (gmt 0)

2:30 pm on Aug 28, 2001 (gmt 0)

Many thanks
Ian

12:17 am on Sep 7, 2001 (gmt 0)

There's also my own page at [jafsoft.com...]

Although it's more concerned with listing agents, it does list some IPs that were in use at the time the agent was added to the page.

3:49 pm on Sep 20, 2001 (gmt 0)

Here are my lists:

6:14 pm on Sep 26, 2001 (gmt 0)

I found a company that offers a downloadable list of spiders and what not at [fantomaster.com...]

Do any of you folks know of any other such services or products?

many Thanks