Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Secure and other valid proxy sources
An escape from servers[3] ( 1 2 3 )
84 dstiles 11:07 pm June 29, 2015
  version 1.7 . of what?
4 lucy24 2:18 am June 27, 2015
  /trackback/
11 lucy24 4:21 pm June 25, 2015
  humanoid Chrome
5 lucy24 10:54 pm June 24, 2015
  SkypeUriPreview
8 lucy24 9:44 pm June 19, 2015
  LinqiaScrapeBot
2 keyplyr 5:36 pm June 17, 2015
  Should I be worried?
Many erroneous requests from Russia/Ukraine
18 Rawrishly 4:56 am June 16, 2015
  Reverter/2.0 & QuickiWiki/1.0
2 keyplyr 9:50 pm June 15, 2015
  Haansoft
10 keyplyr 10:01 am June 13, 2015
  Pocket by Firefox
Browser based social media sharing[2] ( 1 2 )
32 keyplyr 9:06 am June 8, 2015
  Stumbleupon?
7 aristotle 9:14 pm June 6, 2015
  AVG & favicons
4 keyplyr 10:07 pm June 5, 2015
  Is this a human?
18 aristotle 8:13 pm June 4, 2015
  PageGlimpse
Another blatant scraper
13 blend27 5:35 pm June 4, 2015
  Distributed Internet Measurements and Simulations Headquarters
11 aristotle 9:53 am June 2, 2015
  PageAnalyzer
3 lucy24 11:35 pm May 27, 2015
  Arin Whois Ovh
2 wilderness 2:56 pm May 26, 2015
  New Click Adsense
Russian scam bot
23 keyplyr 2:52 am May 26, 2015
  Domain Re-Animator Bot
Zombie apocalypse...
7 trintragula 2:11 am May 23, 2015
  New ip class for googlebot ?
193.235.238.233
3 adresanet 1:16 am May 20, 2015
  Amazon AWS Hosts Bad Bots
Continuation Thread[4] ( 1 2 3 4 )
108 incrediBILL 7:25 am May 18, 2015
  SputnikBot
could be worse...
3 trintragula 7:09 pm May 17, 2015
  Scraping May Be A Biological Imperative
Scraping is actually a TREME according to Stephen Hawking
11 incrediBILL 7:26 am May 16, 2015
  Findxbot
new bot on the hunt
17 dupres01 8:59 pm May 13, 2015
  OmniBot
Microsoft Corp.
12 Pfui 8:06 pm May 13, 2015