Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Need help ID ing a spider
2 RobertS 12:35 pm May 13, 2002
  What is Dsurf?
6 Brett_Tabke 9:41 am May 12, 2002
  Gozilla/4.0
Just looking for formmail.pl
7 bobriggs 9:30 am May 12, 2002
  UniverserseBot/1.0 from 209.17.183.249
NSlookup says thats gw.activestate.com
2 mbauser2 11:21 pm May 11, 2002
  SlySearch/1.0, why is this on my logs?
why would this be on my logs
23 PinkPanther 12:26 pm May 11, 2002
  DIA-Test?
4 keyplyr 6:25 pm May 10, 2002
  ia_archiver?
19 Kamikaze 3:33 pm May 9, 2002
  Detection of Moreover.com Spiders
Does anybody know the UA?
4 Eric_Lander 1:07 pm May 9, 2002
  InfoLink/1.9
3 misosoph 11:36 pm May 8, 2002
  GetBot
5 keyplyr 9:25 pm May 7, 2002
  New Google IP?
5 msgraph 7:20 pm May 7, 2002
  New FAST UA?
3 msgraph 6:55 pm May 6, 2002
  Anyone heard of MOT 28 spider ?
Anyone heard of MOT 28 spider ?
3 xr2boy 4:57 pm May 6, 2002
  What are inktomi's spiders?
2 korkus2000 3:46 pm May 6, 2002
  Mercator?
What is this user agent?
6 przero2 3:26 pm May 6, 2002
  icorus.com
No one seems to have written about this robot
4 misosoph 1:06 pm May 6, 2002
  These guys scour without UA
3 littleman 2:49 am May 5, 2002
  a few new ips and UAs
21 Crazy_Fool 11:35 pm May 4, 2002
  Australian Academic and Research Network
2 wilderness 7:12 am May 4, 2002
  Possible AOL spider detected
Looking for confirmation...
22 Key_Master 1:13 am May 4, 2002
  any idea what spider this IP belongs to?
3 ferrari360 3:51 am May 3, 2002
  fastsearch?
2 johnhamman 7:53 pm May 2, 2002
  Up to date list?
2 Filipe 11:42 pm May 1, 2002
  Inktomi
just added a new Class C
5 volatilegx 3:44 pm Apr 30, 2002
  Need help with blocking this agent
4 willtell 5:41 pm Apr 29, 2002