Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Bingbot with one domain and two IPs
What's the point of that?
10 moTi 3:18 am Nov 16, 2012
CompSpyBot
68.47.129.55
8 Bewenched 9:51 pm Nov 15, 2012
adbeat.com
174.129.111.51
2 Bewenched 10:48 pm Nov 14, 2012
Educational Unit
took 52 files - one image only - the contact image
13 Ken_S 5:17 am Nov 14, 2012
www.integromedb.org/Crawler
2 Bewenched 12:43 am Nov 14, 2012
the art-loving iPad
8 lucy24 6:36 pm Nov 12, 2012
Colocation
4 Ken_S 12:57 pm Nov 12, 2012
Mythic Beasts Test Netblock
6 keyplyr 7:57 pm Nov 11, 2012
Remote host
3 wilderness 10:53 am Nov 11, 2012
Jbot
3 keyplyr 3:03 am Nov 11, 2012
Korea KRNIC
4 Bewenched 9:23 pm Nov 10, 2012
Image thief
176.62.49.50
3 Bewenched 2:38 pm Nov 10, 2012
Proactive vs. Reactive IP Range Blocking
General Policy or Behavior Based
7 incrediBILL 9:36 pm Nov 9, 2012
Amazon motherload
4 Bewenched 11:48 pm Nov 8, 2012
MaxPointCrawler / maxpointinteractive.com
208.123.79.68
4 Bewenched 3:17 pm Nov 7, 2012
Wowrack
10 keyplyr 9:11 am Nov 7, 2012
EasouSpider
4 keyplyr 6:58 am Nov 7, 2012
odd behavior
216.38.216.101
5 Bewenched 6:34 am Nov 7, 2012
Strange Inktomi Corporation IP
11 SEOPTI 1:44 pm Nov 6, 2012
orchestr8
66.35.60.158
5 Bewenched 7:32 am Nov 6, 2012
Does amazon have a crawler or are all Amazon AWS IPs fair game?
9 bigtoga 6:57 am Nov 6, 2012
New google IP range
173.194/16
19 dstiles 1:03 pm Nov 5, 2012
Microblog-Explorer/0.3
Freie Universitaet Berlin
3 keyplyr 8:53 pm Nov 4, 2012
elefent/Elefent 1.1
4 keyplyr 11:45 pm Nov 3, 2012
TotientBot
new bot sighting
3 incrediBILL 10:21 pm Nov 3, 2012