Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Net Depot, Inc.
216.180.241.154
3 photoman 12:55 am July 9, 2003
  UA Change in 12 seconds
4 photoman 10:37 pm July 7, 2003
  Under the Rainbow?
13 SeanL 8:56 pm July 7, 2003
  Batik/1.0
Floral
4 DavidT 6:42 pm July 7, 2003
  GalaxyBot
I have never seen this before
8 quotations 3:26 pm July 7, 2003
  The Big List
Identify the good guys...
3 hanuman 9:09 am July 7, 2003
  217.155.46.193
3 wilderness 6:50 am July 7, 2003
  Seeker.lookseek.com
Seeker.lookseek.com
4 keyplyr 12:39 am July 7, 2003
  Scooter
Not Good
14 wilderness 9:05 pm July 6, 2003
  what spider is this!
and what is it seeing
4 madmax1 5:46 pm July 6, 2003
  alta vista 24 hr update spider?
2 chompy 4:57 pm July 6, 2003
  MediaPartners
JavaScript
6 DavidT 3:15 am July 5, 2003
  New network?
64.68.88.134
4 pontifex 6:50 pm July 4, 2003
  Banning IP for Tripod / Lycos sites
Banning IP for Tripod / Lycos sites
7 HardKnox 3:21 pm July 4, 2003
  New Crawler?
Today a lot of visits from 66.123.16.67
3 angiolo 1:37 pm July 4, 2003
  Dirty Agent Tricks
How to deny
3 nuhkweb 3:15 am July 4, 2003
  Did anybody try Allow in robots.txt?
Allow only Googlebot but no others
4 HitProf 8:26 pm July 3, 2003
  Google hardly!
just a heads up
2 wilderness 5:53 pm July 3, 2003
  tovero
6 wilderness 6:13 am July 3, 2003
  Netscape/PICgrabber
4 keyplyr 12:23 am July 3, 2003
  Zao/0.1
Another resurrected spyder?
14 pendanticist 10:58 pm July 2, 2003
  Univ of Toyko
Just a heads up
2 wilderness 10:53 pm July 2, 2003
  Firefly
2 WebJoe 11:59 am July 2, 2003
  New Excite bot?
Similar IP but no ID
3 too_much_information 12:09 pm July 1, 2003
  grub and ZyBorg
what's going on?
8 claus 8:17 am July 1, 2003