Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Unknown Crawler
193.95.83.130
9 chris_f 8:07 am May 15, 2002
  spider-loh-tc063.proxy.aol.com
What Is This
5 Funfone 8:44 pm May 14, 2002
  AltaVista Spider?
trek17.sv.av.com
3 HyperGeek 4:27 pm May 14, 2002
  Unknown - Lots of activity
66.28.250.176
3 HyperGeek 4:02 pm May 14, 2002
  Two mystery spiders
Heavy crawlers
15 Everyman 5:09 am May 14, 2002
  Need help ID ing a spider
2 RobertS 12:35 pm May 13, 2002
  What is Dsurf?
6 Brett_Tabke 9:41 am May 12, 2002
  Gozilla/4.0
Just looking for formmail.pl
7 bobriggs 9:30 am May 12, 2002
  UniverserseBot/1.0 from 209.17.183.249
NSlookup says thats gw.activestate.com
2 mbauser2 11:21 pm May 11, 2002
  SlySearch/1.0, why is this on my logs?
why would this be on my logs
23 PinkPanther 12:26 pm May 11, 2002
  DIA-Test?
4 keyplyr 6:25 pm May 10, 2002
  ia_archiver?
19 Kamikaze 3:33 pm May 9, 2002
  Detection of Moreover.com Spiders
Does anybody know the UA?
4 Eric_Lander 1:07 pm May 9, 2002
  InfoLink/1.9
3 misosoph 11:36 pm May 8, 2002
  GetBot
5 keyplyr 9:25 pm May 7, 2002
  New Google IP?
5 msgraph 7:20 pm May 7, 2002
  New FAST UA?
3 msgraph 6:55 pm May 6, 2002
  Anyone heard of MOT 28 spider ?
Anyone heard of MOT 28 spider ?
3 xr2boy 4:57 pm May 6, 2002
  What are inktomi's spiders?
2 korkus2000 3:46 pm May 6, 2002
  Mercator?
What is this user agent?
6 przero2 3:26 pm May 6, 2002
  icorus.com
No one seems to have written about this robot
4 misosoph 1:06 pm May 6, 2002
  These guys scour without UA
3 littleman 2:49 am May 5, 2002
  a few new ips and UAs
21 Crazy_Fool 11:35 pm May 4, 2002
  Australian Academic and Research Network
2 wilderness 7:12 am May 4, 2002
  Possible AOL spider detected
Looking for confirmation...
22 Key_Master 1:13 am May 4, 2002