Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Sogou spider? Good or Bad?
2 phi148 3:28 pm Jan 2, 2007
  fake google bot?
6 pp46 11:19 am Jan 2, 2007
  LinkWalker/2.0
Anyone knows what this bot does?
3 ibnuasad 9:29 pm Jan 1, 2007
  bot/1.0 (bot; http://; bot@bot.bot)
7 Mokita 7:41 am Dec 29, 2006
  Yodaobot 1.0 is coming
Do you notice the Outfoxbot 0.x? Last weekend it released as Yodaobot 1.0
16 chedong 6:04 pm Dec 28, 2006
  Exabot-Test/1.0
2 GaryK 6:35 am Dec 25, 2006
  Yahoo invents browser that doesn't need an OS!
Y! sitemap checker
2 jdMorgan 5:05 pm Dec 22, 2006
  IE7 Save Site UA?
7 keyplyr 9:58 am Dec 21, 2006
  User-Agent begins with "User Agent::"
4 Umbra 12:51 am Dec 19, 2006
  WorQmada/1.0
6 bobothecat 6:02 am Dec 18, 2006
  Bigsearch.ca
new Canadian search engine currently Beta
3 Mokita 2:58 am Dec 16, 2006
  MSNBot from APNIC?
Is this a spoofed user-agent?
5 Kadence 7:53 pm Dec 15, 2006
  MSNPTC/1.0 Bot Gone Wild
MSN's AdCenter Bot
4 speakSEM 7:31 pm Dec 14, 2006
  new disguise of rippers?
8 the_nerd 3:37 am Dec 14, 2006
  Analyzing requests on every pageview vs. Every 1 minute
Pros & cons...
8 MichaelBluejay 2:34 am Dec 14, 2006
  HMSE Robot
Not a clue what it is.
3 GaryK 1:02 am Dec 14, 2006
  New Search Engine Spider IPs
4 volatilegx 3:31 pm Dec 13, 2006
  Nokia6600
14 wilderness 10:37 pm Dec 10, 2006
  Yahoo Bots
8 sem4u 11:02 am Dec 6, 2006
  Amazon
3 wilderness 3:32 pm Dec 2, 2006
  Is Ebay attempting wholesale site downloads?
Regular attempts from the EBay Network to download my entire site.
24 AlexK 12:32 am Nov 30, 2006
  masidani bot v0.6(www.masidani.com)
Holiday shopping bot
3 GaryK 12:18 am Nov 27, 2006
  Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete ma
New UA for Naver
3 GaryK 12:15 am Nov 27, 2006
  ICRA label generator
ICRA_Semantic_spider
6 GaryK 8:34 pm Nov 26, 2006
  SirSeek
badly malformed UA
2 GaryK 8:07 pm Nov 26, 2006