Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Nokia bad behaviour
Is this a cloud, a bot or a proxy?
3 dstiles 6:36 pm June 2, 2012
  New Blekkobot User Agent String
scoutjet / blekkobot
11 dstiles 9:04 pm May 30, 2012
  Bing plain clothes
obnoxious pest
2 Staffa 12:57 am May 30, 2012
  Spider Forum Thread Titles
16 incrediBILL 10:17 pm May 29, 2012
  Bad Spiders and Proxy Servers
Using Opera Proxy
19 grandma_genie 7:25 am May 27, 2012
  HEAD request for favicon
"Your favicon? What would it want with me?"
2 lucy24 2:44 am May 25, 2012
  url include
12 wilderness 8:40 pm May 23, 2012
  "Capture Extractor" Botnet Crawling Observed
2 incrediBILL 3:48 pm May 23, 2012
  Injection Attempt From Microlink IPs
4 keyplyr 8:59 pm May 22, 2012
  Scrapers Using Peak Web Hosting and Generic Browser User Agents
4 keyplyr 10:53 pm May 19, 2012
  deny all
15 wilderness 3:50 pm May 14, 2012
  Yahoo SE
2 wilderness 1:46 pm May 14, 2012
  Bing blows a gasket
9 wilderness 7:57 pm May 13, 2012
  PycURL
4 wilderness 7:00 pm May 13, 2012
  Cookie based spider catcher
6 motorhaven 11:16 pm May 12, 2012
  Been getting 200+ page loads per day from same IP
recorded by js based analytics
10 scooterdude 7:01 pm May 12, 2012
  Digital AlphaServer User Agent
3 wilderness 6:26 pm May 12, 2012
  Bright House Networks
3 wilderness 11:21 pm May 10, 2012
  HopOne
10 keyplyr 9:17 pm May 10, 2012
  MediaTemple
3 keyplyr 8:58 pm May 10, 2012
  Multiple User Agents, same IP
Is this a bot?
17 grandma_genie 8:49 pm May 10, 2012
  User Agent "-"
missing User Agent
8 dupres01 8:01 pm May 9, 2012
  meet the pdfbot
New Variation of Googlebot?
3 lucy24 7:48 pm May 9, 2012
  More Multi-National Attempts from Avant Browser and IE 5 on Win 3.1
11 incrediBILL 8:41 pm May 8, 2012
  The Onion Strikes - TOR Proxy Comment Spammers
tor exit node drive-by
9 iamzippy 6:57 pm May 3, 2012