Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  User Agent "-"
missing User Agent
8 dupres01 8:01 pm May 9, 2012
  meet the pdfbot
New Variation of Googlebot?
3 lucy24 7:48 pm May 9, 2012
  More Multi-National Attempts from Avant Browser and IE 5 on Win 3.1
11 incrediBILL 8:41 pm May 8, 2012
  The Onion Strikes - TOR Proxy Comment Spammers
tor exit node drive-by
9 iamzippy 6:57 pm May 3, 2012
  open proxies
7 wilderness 12:12 pm May 3, 2012
  MojeekBot
14 wilderness 2:16 pm May 2, 2012
  fatlens.com using Googlebot agents?
5 brokaddr 1:19 pm May 2, 2012
  Besthosting
3 keyplyr 9:46 pm Apr 29, 2012
  EGIHosting
4 keyplyr 2:20 pm Apr 28, 2012
  WeblexBot
8 keyplyr 7:26 pm Apr 27, 2012
  *missing* - user agent from Yahoo, or spoof?
7 brokaddr 3:13 am Apr 27, 2012
  LitlrBot
mobile SE
2 keyplyr 1:16 am Apr 27, 2012
  AWS Ireland?
12 incrediBILL 5:59 am Apr 22, 2012
  more colo's
5 wilderness 1:56 am Apr 22, 2012
  Multi-National Opera 8 invasion
11 incrediBILL 8:49 am Apr 21, 2012
  Google search without keywords valid
6 motorhaven 5:06 am Apr 21, 2012
  HEAD checks
3 wilderness 3:56 am Apr 20, 2012
  AskTb redirect
4 wilderness 3:40 am Apr 20, 2012
  Speakeasy
4 wilderness 9:53 pm Apr 19, 2012
  Electricfiber.net
Nuance/Tegic Linguistics Team
4 keyplyr 9:01 pm Apr 17, 2012
  What is bot[+:,\.\;\/\\-]?
7 ichthyous 10:17 pm Apr 16, 2012
  Jakarta Commons
3 wilderness 8:24 pm Apr 16, 2012
  ia archiver
random hits from China
19 dstiles 9:14 pm Apr 15, 2012
  EventMachine
3 wilderness 9:05 pm Apr 14, 2012
  "\"Mozilla/5.0"
8 keyplyr 10:11 am Apr 13, 2012