Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  can anyone id this spider
3 nube 12:41 am Dec 9, 2001
  Whew. iaea.org is hyper
10 toolman 12:10 am Dec 9, 2001
  Larbin
Starting to see more of them...
3 netcommr 11:58 pm Dec 8, 2001
  ArchitextValidator
what for?
2 bobriggs 10:27 pm Dec 6, 2001
  Mozilla/2.0 (compatible; Ask Jeeves)
With a referer?
13 bobriggs 10:20 pm Dec 6, 2001
  Is this a new spider
pingalink
4 Ove 8:07 pm Dec 3, 2001
  Googlebot Spider
11 Jill 7:08 pm Dec 3, 2001
  Odd Useragents
11 Will 9:14 am Dec 3, 2001
  New Scooter UA from AV
2 msgraph 7:52 pm Nov 30, 2001
  Strange UA
5 BoneHeadicus 5:15 pm Nov 25, 2001
  Another new Scooter UA
209.73.162.151 Scooter_trk15-3.0.3
11 Crazy_Fool 11:40 pm Nov 24, 2001
  MixCat and WebFind
two search engines I just came across
3 volatilegx 4:09 am Nov 23, 2001
  Identifying off-line browsers
3 mikeworld 5:38 pm Nov 22, 2001
  Spider Simulator software
The simulator is not finding keywords in my page. What's wrong?
7 sunsetsands 5:17 am Nov 16, 2001
  Xyleme, Inria, Cosmos and Sqworm
They are related
7 volatilegx 4:41 pm Nov 15, 2001
  Looking for info on the following possible spiders
10 volatilegx 2:53 pm Nov 13, 2001
  Yet another email collector
Light Sense Inc HTTP Control
2 bobriggs 4:19 pm Nov 12, 2001
  PCBeaconBot/1.0
a new one for me
4 idiotgirl 2:58 pm Nov 10, 2001
  Spider analyzer
application which analyses se-spiders and compare them with a list...
2 heini_dutch 4:52 pm Nov 9, 2001
  Scooter-ARS-1.1-ih
5 littleman 8:23 pm Nov 8, 2001
  Cannot get to search engines. Funny or stupid
None of the search engines goes further default.asp page
8 Andrewb 2:56 pm Nov 7, 2001
  Flow of Robots and Spiders
3 agerhart 2:23 pm Nov 7, 2001
  PortalBSpider
6 Leer555 3:17 pm Nov 6, 2001
  5 fold increase in daily hits due to 195.8.179.170
Its being very abusive but I dont know who it is
5 Robber 6:13 pm Nov 4, 2001
  Spider User Agents
ID via user agents
5 phrequency 10:45 am Nov 1, 2001