Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  209.49.118.22
2 wilderness 6:42 pm May 19, 2003
  New robot?
3 fandangood 6:00 pm May 19, 2003
  Java Script and DHTML Spider Detection
Is text detected by the spiders
6 jgold454 8:34 pm May 17, 2003
  CJ Spider
?
16 DavidT 4:40 pm May 17, 2003
  65.94.226.152
new spider?
11 werty 6:37 am May 17, 2003
  AOL search query results in single hit to page
requests only html file w/o menu, css, background, etc.
7 nativenewyorker 8:42 pm May 16, 2003
  Here's a new one...
Microsoft Internet Browser
3 volatilegx 5:15 pm May 16, 2003
  New Websense IP?
64.156.198.69
4 WitchLars 3:58 am May 16, 2003
  The BanBots Project site
anyone knows what happened to it?
7 hanuman 3:49 am May 16, 2003
  New spider? "HTML Works 5.5"
Could not find any reference to this anywhere else so thought I'd ask here.
6 bugblatter 2:21 am May 16, 2003
  NPBot?
and this one is?
4 wkitty42 4:36 pm May 15, 2003
  FTAgent
Anyone seen this one?
3 mvl22 2:14 pm May 15, 2003
  Program Shareware Spider
What is it?
2 Bio4ce 12:54 am May 15, 2003
  GrigorBot 0.8
5 WarmGlow 11:14 pm May 14, 2003
  Spider List
spiders, robots
3 seolist 10:48 pm May 14, 2003
  Odd Spider
Received an unknown spider
4 Strange 8:37 pm May 14, 2003
  Some unique visitors
6 wilderness 5:00 pm May 14, 2003
  Googleperson?
216.239.37.5
3 coconutz 1:25 pm May 14, 2003
  can anyone identify this
ASPseek/1.2.10
2 indiandomain 6:31 am May 14, 2003
  AsiaNetBot
anyone knows this one?
3 bull 11:40 pm May 13, 2003
  SurveyBot/2.2
6 elektra 6:56 pm May 13, 2003
  ask.24x.info No Crawler Name
62.75.136.40
2 jim_w 12:52 pm May 13, 2003
  Anyone heard of this one
tide72.microsoft.com
2 totter 7:53 am May 13, 2003
  blocking Road Runner
10 cayleyv 9:37 am May 10, 2003
  Enticing a spider to crawl the site
Robots.txt or no Robots.txt
6 SinclairUser 1:01 am May 10, 2003