Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Can we run servlets on apache web server
2 jolly 3:13 pm Mar 13, 2001
  63.73.85.3
2 web_t 6:40 am Mar 11, 2001
  Alltheweb spider
5 stn24 5:56 am Mar 11, 2001
  Speedy Spider
8 VAL@Amsterdam 7:32 pm Mar 8, 2001
  Msn
new to this
2 wayne 6:02 pm Mar 8, 2001
  Sneaky
2 BoneHeadicus 5:22 pm Mar 8, 2001
  Very annoying robot
18 msgraph 4:16 pm Mar 7, 2001
  Whatsup wit 24/7
3 BoneHeadicus 9:00 pm Mar 4, 2001
  Didn't get stung by the Jellyfish1
2 BoneHeadicus 2:48 am Mar 4, 2001
  dumb spider: asterias/2.0
coho.singingfish.com: asterias/2.0
8 skirril 8:32 pm Mar 2, 2001
  New Altavista-spider?
testing new scooter and mercator or what else?
7 oLeon 4:05 pm Mar 2, 2001
  New GoogleBot
13 BoneHeadicus 2:33 pm Mar 2, 2001
  Great lookup tool
Great program
6 tcmalaska 7:14 pm Mar 1, 2001
  Concentric Network Corporation
4 msgraph 2:15 pm Mar 1, 2001
  SnoopRob
Where does SnoopRob come from?
3 pikaia 7:41 am Mar 1, 2001
  User-Agent: MyLycos
Please, this is all over my stuff
16 han solo 7:25 pm Feb 28, 2001
  Ip:199.245.188.74
UA: libwww-perl/5.50
4 roscoepico 6:43 pm Feb 28, 2001
  who is this?
5 hohmaster 5:30 pm Feb 28, 2001
  Newbie - Do Spiders eat JPG files?
I could not identify this IP
8 Macguru 12:34 pm Feb 28, 2001
  Who/What is this?
2 jk321 8:09 am Feb 27, 2001
  Crawler6.bos2.fast-search.net
FAST / Alltheweb crawler?
5 Rumbas 5:28 am Feb 25, 2001
  Make them stop!
any luck?
8 luckynh 1:46 pm Feb 21, 2001
  211.117.61.68
4 Froggyman 8:45 pm Feb 20, 2001
  Sidewinder
What is it doing?
3 Hope 6:50 pm Feb 20, 2001
  JennyBot/0.1?
64.97.116.103; JennyBot/0.1
5 skirril 2:27 am Feb 20, 2001