Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  ebay Germany
4 topr8 5:21 am Sept 8, 2012
  MASP in IE 9 user agent
5 yaashul 7:42 am Sept 6, 2012
  Stop the presses - a legit user from an Amazon range
3 motorhaven 7:44 pm Sept 5, 2012
  bingbot joins the 131.253 party
6 lucy24 8:48 pm Sept 3, 2012
  \x95Mozilla/4.0
2 keyplyr 8:39 pm Sept 3, 2012
  FHScan Core
What in the world is this?
2 grandma_genie 1:54 am Aug 31, 2012
  MSN Bot
17 wilderness 6:23 pm Aug 29, 2012
  Twitter for Testing Bot Blocking
5 incrediBILL 6:04 am Aug 29, 2012
  cache request
3 wilderness 2:43 pm Aug 26, 2012
  chlooe
SEO service
11 keyplyr 1:39 pm Aug 26, 2012
  imrd/1.0
another SEO bot
2 incrediBILL 12:40 pm Aug 25, 2012
  Selenium Server Question
What is the purpose of this type of activity...
15 grandma_genie 4:48 am Aug 24, 2012
  The Idaho crawlers...
I wonder who's behind this
12 bakedjake 7:24 pm Aug 23, 2012
  Opera (Ukraine Edition)
Odd Substring In User Agent
4 iamzippy 9:36 pm Aug 21, 2012
  Yandex.ru now running yandex.com spider
Wanted to let others know they may want to update their black/white list.[2] ( 1 2 )
46 JAB_Creations 11:58 pm Aug 20, 2012
  British Library scraper
bl.uk_lddc_bot/3.1.1
6 dstiles 7:01 pm Aug 16, 2012
  Bots by Hosting Company
6 incrediBILL 2:51 am Aug 16, 2012
  SEOENGWorldBot
3 wilderness 12:43 am Aug 16, 2012
  news bot
prateh.ru
8 keyplyr 8:04 pm Aug 13, 2012
  google and favicon
strange google behavior
7 dupres01 11:40 am Aug 13, 2012
  bottlenose.com
5 keyplyr 7:49 pm Aug 12, 2012
  Something from Myspace IP
5 motorhaven 7:27 pm Aug 9, 2012
  Blekko/Scoutjet
10 keyplyr 4:21 am Aug 4, 2012
  opera-mini.net bot tagting /feed address how to redirect it
bot hits /feed address using hsot name opera-mini.net
9 SaleF 6:08 pm Aug 2, 2012
  131.253.21.0 - 131.253.47.255 msnbots?
I got a hit from an msnbot within this range, is this normal?
3 MickeyRoush 1:47 pm Aug 2, 2012