Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Bingbot with one domain and two IPs
What's the point of that?
10 moTi 3:18 am Nov 16, 2012
  CompSpyBot
68.47.129.55
8 Bewenched 9:51 pm Nov 15, 2012
  adbeat.com
174.129.111.51
2 Bewenched 10:48 pm Nov 14, 2012
  Educational Unit
took 52 files - one image only - the contact image
13 Ken_S 5:17 am Nov 14, 2012
  www.integromedb.org/Crawler
2 Bewenched 12:43 am Nov 14, 2012
  the art-loving iPad
8 lucy24 6:36 pm Nov 12, 2012
  Colocation
4 Ken_S 12:57 pm Nov 12, 2012
  Mythic Beasts Test Netblock
6 keyplyr 7:57 pm Nov 11, 2012
  Remote host
3 wilderness 10:53 am Nov 11, 2012
  Jbot
3 keyplyr 3:03 am Nov 11, 2012
  Korea KRNIC
4 Bewenched 9:23 pm Nov 10, 2012
  Image thief
176.62.49.50
3 Bewenched 2:38 pm Nov 10, 2012
  Proactive vs. Reactive IP Range Blocking
General Policy or Behavior Based
7 incrediBILL 9:36 pm Nov 9, 2012
  Amazon motherload
4 Bewenched 11:48 pm Nov 8, 2012
  MaxPointCrawler / maxpointinteractive.com
208.123.79.68
4 Bewenched 3:17 pm Nov 7, 2012
  Wowrack
10 keyplyr 9:11 am Nov 7, 2012
  EasouSpider
4 keyplyr 6:58 am Nov 7, 2012
  odd behavior
216.38.216.101
5 Bewenched 6:34 am Nov 7, 2012
  Strange Inktomi Corporation IP
11 SEOPTI 1:44 pm Nov 6, 2012
  orchestr8
66.35.60.158
5 Bewenched 7:32 am Nov 6, 2012
  Does amazon have a crawler or are all Amazon AWS IPs fair game?
9 bigtoga 6:57 am Nov 6, 2012
  New google IP range
173.194/16
19 dstiles 1:03 pm Nov 5, 2012
  Microblog-Explorer/0.3
Freie Universitaet Berlin
3 keyplyr 8:53 pm Nov 4, 2012
  elefent/Elefent 1.1
4 keyplyr 11:45 pm Nov 3, 2012
  TotientBot
new bot sighting
3 incrediBILL 10:21 pm Nov 3, 2012