Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  What Controls Reverse DNS lookup on Your server?
Probably something out of our control.
7 blend27 9:27 pm Feb 16, 2016
  RBot
10 w3bmastine 11:49 pm Feb 14, 2016
  SafeSearch microdata crawler
safesearch.avira.com seems to have a SafeSearch microdata crawler.
2 w3bmastine 5:36 am Feb 14, 2016
  The Donald web spider user agent
it had to happen.
18 lucy24 10:38 pm Feb 13, 2016
  Searchie
4 keyplyr 6:28 pm Feb 10, 2016
  apple.bot
massive requests
15 johnhh 11:07 pm Feb 7, 2016
  Oddly suspicious traffic claiming duckduckgo
12 grouchy_sysadmin 4:39 am Feb 7, 2016
  BellPagesCA
4 w3bmastine 4:15 am Feb 7, 2016
  Putin
4 keyplyr 3:20 am Feb 6, 2016
  YYSpider
3 keyplyr 10:23 am Feb 4, 2016
  Experibot v1
[2] ( 1 2 )
34 keyplyr 12:33 pm Feb 2, 2016
  Where am I?
12 lucy24 3:30 am Feb 2, 2016
  etoolsbot
4 keyplyr 10:11 pm Jan 26, 2016
  startmebot
3 keyplyr 11:01 pm Jan 20, 2016
  Friendica 'Lily of the valley'
2 keyplyr 9:18 pm Jan 20, 2016
  Exploratodo
2 keyplyr 2:07 pm Jan 20, 2016
  Plukkie
Plukkie is back, acting strange
2 w3bmastine 2:33 am Jan 19, 2016
  VizzitSpider
2 keyplyr 2:27 am Jan 19, 2016
  List of Data Centre IP ranges for blocking purposes
8 Xpat 3:55 am Jan 17, 2016
  SEOdiver
6 keyplyr 10:18 pm Jan 16, 2016
  Svven-Summarizer
6 keyplyr 2:19 am Jan 16, 2016
  wpif and imgsizer
9 lucy24 9:18 am Jan 14, 2016
  EuripBot
3 keyplyr 8:52 am Jan 9, 2016
  golabbot
2 keyplyr 7:13 pm Jan 6, 2016
  vebidoobot
3 keyplyr 6:08 pm Jan 4, 2016