Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Apple
6 wilderness 2:19 am Jan 9, 2013
  browser versions
3 wilderness 1:39 am Jan 9, 2013
  Netscape
5 wilderness 10:15 pm Jan 8, 2013
  Entanet
Entanet-adsl
2 not2easy 10:05 pm Jan 8, 2013
  Undefined Error in Logs
[2] ( 1 2 )
39 yaashul 5:28 am Jan 7, 2013
  One for the UA profilers
5 Bewenched 6:38 am Jan 6, 2013
  Microsoft or not?
70.37.130.8
9 Bewenched 5:56 am Jan 5, 2013
  Yandex translate
4 lucy24 6:36 pm Jan 3, 2013
  that Yahoo thingy
2 wilderness 6:13 am Dec 28, 2012
  Anyone know the name of Wayback Machine's robot?
One site has not been crawled all year... wondering why.
19 JeffOstroff 8:46 pm Dec 27, 2012
  Human or robot?
another one for the profilers
4 lucy24 9:09 am Dec 27, 2012
  Ning/1.0
eat my 403
15 Bewenched 2:51 am Dec 27, 2012
  TosCrawler
14 lucy24 6:55 pm Dec 25, 2012
  WIndows NT 9.0 - Yeeeehaaa
Back from the future!
6 blend27 10:54 pm Dec 23, 2012
  Google scraping through China?
7 dstiles 8:13 pm Dec 20, 2012
  Data reseller bot shortlist?
5 Sgt_Kickaxe 5:40 pm Dec 14, 2012
  'artviper(tm)
85.25.9.83
8 Bewenched 8:43 pm Dec 13, 2012
  Microsoft- really?
IP indicates microsoft, UA is strange
4 dupres01 8:23 pm Dec 13, 2012
  Cloaked yahoo crawler
6 Bewenched 5:41 am Dec 12, 2012
  QippoBot/1.4+(+http://www.qippo.com/bot/)
[2] ( 1 2 )
34 Bewenched 7:31 am Dec 11, 2012
  DomainVader/1.0
3 MxAngel 9:42 pm Dec 10, 2012
  CityGrid.com URL Validation
107.22.0.185
7 Bewenched 8:28 pm Dec 6, 2012
  findlinks bot
130.83.167.153
4 Bewenched 8:00 am Dec 4, 2012
  Yasni
10 incrediBILL 3:03 pm Dec 3, 2012
  Site Rating System
3 Ken_S 2:22 am Dec 3, 2012