Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  webcollage/1.xx
What's the point?
5 LeBain 7:47 pm Oct 31, 2001
  New Excite Spider IP
4 msgraph 7:23 pm Oct 31, 2001
  Software for logs
Want to know when spiders visit - best software?
3 northamptonone 6:49 pm Oct 30, 2001
  Spider ID tools, and beyond
way beyond!
3 awoyo 2:16 am Oct 30, 2001
  Crescent Internet ToolPak HTTP OLE Control v.1.0
Crazy rapid fire bot with bad links!
5 LeBain 5:05 pm Oct 26, 2001
  Nimda as Slurp
2 Josk 8:46 am Oct 26, 2001
  Identify a Connecticut spider
Never saw one like this so thought I would ask
4 Winston 10:37 pm Oct 24, 2001
  cloaked google spider
3 PeteU 7:47 pm Oct 24, 2001
  Amanda_Spider/billy@al-info.com
Rapid fire, but apparently legit seach engine bot
2 LeBain 7:33 pm Oct 24, 2001
  ! New Slurp !
7 toolman 8:50 pm Oct 23, 2001
  MsFrontpage
5 toolman 9:55 pm Oct 22, 2001
  Spider databases?
Downloadable databases with spider info?
4 aetjansen 5:49 pm Oct 22, 2001
  Anybody care to add any bots to this?
17 BoneHeadicus 2:27 pm Oct 22, 2001
  http://169.207.41.120/login.asp
4 jeremy_goodrich 10:11 pm Oct 21, 2001
  Anyone familiar with Searchhippo.com?
Fluffy the Spider!??
6 Travoli 8:21 pm Oct 17, 2001
  Browser=Hunter/3.0 OS=undefined
Requesting totally unrelated URLs....
5 mivox 7:32 pm Oct 17, 2001
  Tool for Checking spider to index my web
I want tool for checking spiders index my website
3 maimax 5:49 pm Oct 17, 2001
  Oliver Perry and w3matter paid a visit
18 Marcia 8:44 pm Oct 16, 2001
  User agent "-" (null)
Are there any legitimate agents that use "-" (null) only?
5 seifried 8:28 pm Oct 15, 2001
  anyone seen this one
217.27.131.25
4 dolphin 1:39 pm Oct 15, 2001
  Incy Wincy?
Looking for some information on this one...
10 Eric_Lander 9:47 am Oct 15, 2001
  Lycos Spider?
Traces to Korea
4 Will 3:39 pm Oct 11, 2001
  Indy Library
6 Marcia 3:27 pm Oct 11, 2001
  ScoreBot/5
6 Son_House 11:03 pm Oct 10, 2001
  New NorthernLight bot
2 littleman 1:44 am Oct 10, 2001