Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  doondbot
2 keyplyr 5:30 am Oct 12, 2010
  Findshare bot
Found this Russian? bot hitting my logs today.
12 grandma_genie 12:14 am Oct 12, 2010
  Yahoo Nokia Symbian crawler
Is this a Yahoo crawler for mobile aps?
3 grandma_genie 1:33 am Oct 9, 2010
  Catchbot
Fetches robots.txt now
3 Dijkgraaf 8:44 am Oct 7, 2010
  sixapart using libwww-perl
5 Pfui 5:34 pm Oct 5, 2010
  Visitor pretending to be Google bot?
Who is gandalf.volutionmedia.com?
16 grandma_genie 1:48 am Oct 3, 2010
  Sosospider
Update on this spider
5 Dijkgraaf 2:41 pm Oct 1, 2010
  123People
4 Dijkgraaf 11:20 am Oct 1, 2010
  Is this a real MSN bot or a fake?
IP seems OK, but is overly zealous.
12 grandma_genie 6:59 pm Sept 29, 2010
  TwengaBot
19 topr8 12:44 am Sept 29, 2010
  Fake Slurp
but referring from Google?
8 Dijkgraaf 10:04 pm Sept 28, 2010
  C4pc
12 Pfui 9:32 pm Sept 28, 2010
  Windows Phone OS 7.0 handset
3 GaryK 1:33 am Sept 28, 2010
  Vnetus? Is this a bot?
5 geekdogfl 10:55 am Sept 27, 2010
  casper bot search attempts to infect sites
user agent changing almost daily[2] ( 1 2 )
41 dstiles 11:30 pm Sept 25, 2010
  Imperia-LinkSpider
Another experimental web bot from Germany
3 Dijkgraaf 2:31 am Sept 24, 2010
  bigfinder
Bad German bot
2 Dijkgraaf 11:54 am Sept 23, 2010
  MSN Bot Spoofing UA as Browser
3 Web_Savvy 3:19 am Sept 23, 2010
  Opera
3 Dijkgraaf 2:13 am Sept 23, 2010
  SolomonoBot/1.01
4 MxAngel 10:00 pm Sept 22, 2010
  BT-ctrlaltdel and Creative ZENcast
Any info on this user agent?
8 grandma_genie 12:06 am Sept 21, 2010
  LinkedInBot
8 Pfui 8:23 pm Sept 20, 2010
  What IPs will Googlebot access my site from?
2 DiscoStu 2:32 pm Sept 20, 2010
  AppEngine-Google: proxy1china
11 Umbra 1:14 pm Sept 18, 2010
  What is wmtips.com?
Possible hacker using this site?
3 grandma_genie 8:04 pm Sept 17, 2010