Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  184.105.140.
Hurricane Electric -- a.k.a. Google Quality Rater?
9 Pfui 8:48 pm Nov 16, 2011
  WASALive
4 Pfui 6:33 pm Nov 16, 2011
  207.38.118.
BAT's Bad Bot(s)
4 Pfui 9:54 pm Nov 15, 2011
  Spider Tracking Links - Examining 2 Methods
13 incrediBILL 10:09 pm Nov 14, 2011
  ColoSpace, Inc.
4 keyplyr 9:06 pm Nov 14, 2011
  siteintel.net
Bot-running bad Opera
8 Pfui 8:44 pm Nov 14, 2011
  Monthly Robots.txt Request From securityspace.com
5 incrediBILL 7:29 pm Nov 11, 2011
  followthatpage
Not on my site!
8 Mokita 7:27 am Nov 9, 2011
  Voxel Cloud
3 keyplyr 11:44 pm Nov 8, 2011
  Adhost
3 keyplyr 11:41 pm Nov 8, 2011
  domainsoutlook.com good, you may rely on it
see magic 8 ball for more information
3 incrediBILL 4:29 am Nov 7, 2011
  robot97, robot99, please report to statshow.com
oh goody it's on godaddy!
6 incrediBILL 1:48 am Nov 7, 2011
  zitetrendz.com izt vatching you!
3 incrediBILL 12:26 am Nov 7, 2011
  hostnology.in's domain scraper site
9 incrediBILL 12:24 am Nov 7, 2011
  dawhois.com and their qwest for my site data
2 incrediBILL 11:26 pm Nov 6, 2011
  tweetmeme scraping update
2 incrediBILL 11:14 pm Nov 6, 2011
  clearwebstats.com and mostly sunny
2 incrediBILL 11:11 pm Nov 6, 2011
  whoare.us who who, who who!
we know who you are too!
3 incrediBILL 10:43 pm Nov 6, 2011
  Influencebot
2 tangor 10:24 pm Nov 6, 2011
  w3who.net's hidden crawler, crouching user agent
7 incrediBILL 10:21 pm Nov 6, 2011
  PagePeeker
2 Pfui 9:43 pm Nov 5, 2011
  unister-bot
4 keyplyr 9:41 pm Nov 5, 2011
  domain tools, surveybot, IP update
3 incrediBILL 2:16 am Nov 5, 2011
  shell.com
2 Pfui 7:23 pm Nov 3, 2011
  GlueText
7 Pfui 7:10 pm Nov 3, 2011