Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  V20xRmRRPT0K
Anyone seen this in stats? Strange Bot?
17 didibreakit 11:48 pm June 15, 2011
  Pfui
Finally heard from her.
6 GaryK 4:28 pm June 13, 2011
  Is this a botnet attack?
admin/file_manager.php/login.php
23 grandma_genie 9:17 pm June 7, 2011
  Need some advices and clarifications
6 macas 12:44 pm June 2, 2011
  How Often To Check Robots.txt?
2 devitnow 7:28 pm June 1, 2011
  Yahoo crawler(s) requesting mangled URLs
2 Jonesy 8:54 pm May 28, 2011
  upload.php
8 wilderness 9:56 pm May 27, 2011
  cmsworldmap dot com
4 tangor 9:19 pm May 26, 2011
  SetEnvIf Remote-Addr
12 wilderness 12:01 am May 26, 2011
  Google as the referrer but no query string
9 agent_x 9:14 pm May 25, 2011
  And Now Google's Doing It. JS Stats Show GoogleBot
[3] ( 1 2 3 )
74 TheMadScientist 10:04 pm May 18, 2011
  not sure
6 wilderness 9:53 pm May 11, 2011
  Mikrotik HttpProxy
Misconfigured headers /scraper
3 caribguy 10:03 pm May 6, 2011
  Adaptive Spider Technology
3 topr8 11:02 pm May 4, 2011
  Microsoft Bot Processing jQuery?
6 TheMadScientist 11:56 pm May 1, 2011
  Chinese scrapers
caught ya
2 Bewenched 6:43 pm Apr 30, 2011
  Exalead - massive spidering
I know this one has been around a while
6 Bewenched 3:25 pm Apr 29, 2011
  Geo-Targetting by IP and Search engines spiders
4 CrazyBigGaz 7:40 pm Apr 28, 2011
  literate robots?
14 lucy24 1:40 am Apr 27, 2011
  who are marshal dot com and what do they want?
7 lucy24 6:48 pm Apr 20, 2011
  Indonesian scraper
bad scraper go away.
3 Bewenched 7:12 am Apr 20, 2011
  more MS
3 wilderness 11:59 pm Apr 19, 2011
  Odd
4 wilderness 9:07 pm Apr 15, 2011
  Comment spammer?
5 CyberShadow 7:54 pm Apr 15, 2011
  the return of spbot
continuing the spbot / seoprofiler discussion
3 lucy24 10:09 pm Apr 9, 2011