Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Cowdog Bot
Head 'em up and move 'em out. Rawhide
2 Josk 2:53 pm Nov 4, 2003
  alexa
new spider to me
6 dgdclynx 12:08 pm Nov 4, 2003
  UA "MSIE 6.0" (and nothing else)
hit the trap, masquerading bot?
12 WebJoe 8:39 am Nov 4, 2003
  Cowbot 0.1
5 BlueSky 8:21 am Nov 4, 2003
  Is this an invalid UA?
Mozilla/6.0 (compatible; MSIE 6.0; Windows NT 5.2)
7 Wizcrafts 6:39 am Nov 4, 2003
  Spider masking as a UCMore toolbar?
"GET /favicon.ico HTTP/1.1" 200 - "-" "UCmore"
21 cyberkat 4:48 am Nov 4, 2003
  infomine.ucr.edu
at apnic ip?
6 bull 2:55 am Nov 4, 2003
  Can I ban unwanted bots without Apache?
My ISP tells me they run Zeus
5 Michael_Anthony 10:12 pm Nov 3, 2003
  66.232.17.78
fell in bot trap...and a question
9 coyote 5:46 pm Nov 3, 2003
  Mac Finder / rima-tde.net
Spider from Spanish spam friendly ISP?
2 jmccormac 4:57 pm Nov 3, 2003
  vang.net spider 1.6 (Spider 1.7/site@vang.net)
vang.net + soegning.dk - aggressive new spider
2 claus 11:44 am Nov 3, 2003
  Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com)
218.145.25.105 (Kornet, Naver)
2 claus 11:21 am Nov 3, 2003
  P.Arthur 1.1
Are you Bot or not?
10 Gopi_Sundaram 10:57 am Nov 3, 2003
  Blank UA and referer with googlebot IPs
all asking for image files
2 nancyb 6:56 am Nov 3, 2003
  206.16.1.x (cnet) killing us
5 papamaku 2:03 am Nov 3, 2003
  Expired Domain Sleuth
Nuther gnew won?
5 pendanticist 2:19 pm Nov 2, 2003
  How NOT to circulate your resume
(tsk, tsk, tsk.)
4 pendanticist 12:42 pm Nov 2, 2003
  38.112.195.45 -- fell in bot trap
3 BlueSky 12:16 pm Nov 2, 2003
  Strange Googlebot IP
203.199.124.xxx
5 plasma 6:55 am Nov 2, 2003
  216.185.57.106
fell in bot trap
3 coyote 5:44 am Nov 2, 2003
  Google log: StripIt 0.2
Who is this?
5 Jesse_Smith 4:31 am Nov 2, 2003
  Unknown bot ...
Anyone know who is this?
6 ideavirus 4:21 am Nov 2, 2003
  YahooSeeker/1.0
19 akogo 3:28 am Nov 2, 2003
  da 4.0
2 wilderness 10:12 pm Nov 1, 2003
  207.193.137.16
Comes with no UA or referer.
2 addvisors 11:27 am Nov 1, 2003