Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Yodaobot 1.0 is coming
Do you notice the Outfoxbot 0.x? Last weekend it released as Yodaobot 1.0
16 chedong 6:04 pm Dec 28, 2006
  Exabot-Test/1.0
2 GaryK 6:35 am Dec 25, 2006
  Yahoo invents browser that doesn't need an OS!
Y! sitemap checker
2 jdMorgan 5:05 pm Dec 22, 2006
  IE7 Save Site UA?
7 keyplyr 9:58 am Dec 21, 2006
  User-Agent begins with "User Agent::"
4 Umbra 12:51 am Dec 19, 2006
  WorQmada/1.0
6 bobothecat 6:02 am Dec 18, 2006
  Bigsearch.ca
new Canadian search engine currently Beta
3 Mokita 2:58 am Dec 16, 2006
  MSNBot from APNIC?
Is this a spoofed user-agent?
5 Kadence 7:53 pm Dec 15, 2006
  MSNPTC/1.0 Bot Gone Wild
MSN's AdCenter Bot
4 speakSEM 7:31 pm Dec 14, 2006
  new disguise of rippers?
8 the_nerd 3:37 am Dec 14, 2006
  Analyzing requests on every pageview vs. Every 1 minute
Pros & cons...
8 MichaelBluejay 2:34 am Dec 14, 2006
  HMSE Robot
Not a clue what it is.
3 GaryK 1:02 am Dec 14, 2006
  New Search Engine Spider IPs
4 volatilegx 3:31 pm Dec 13, 2006
  Nokia6600
14 wilderness 10:37 pm Dec 10, 2006
  Yahoo Bots
8 sem4u 11:02 am Dec 6, 2006
  Amazon
3 wilderness 3:32 pm Dec 2, 2006
  Is Ebay attempting wholesale site downloads?
Regular attempts from the EBay Network to download my entire site.
24 AlexK 12:32 am Nov 30, 2006
  masidani bot v0.6(www.masidani.com)
Holiday shopping bot
3 GaryK 12:18 am Nov 27, 2006
  Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete ma
New UA for Naver
3 GaryK 12:15 am Nov 27, 2006
  ICRA label generator
ICRA_Semantic_spider
6 GaryK 8:34 pm Nov 26, 2006
  SirSeek
badly malformed UA
2 GaryK 8:07 pm Nov 26, 2006
  "CMS crawler"
their URL gives "Access Denied"
6 Mokita 7:04 pm Nov 26, 2006
  Brandimensions
5 mkaresh 7:01 am Nov 26, 2006
  dearch/0.7.1 (suchmaschine; www.google.de; ich@hier.de)
Another Google spoofer?
3 GaryK 7:22 pm Nov 21, 2006
  Cyveillance
time for an updated ip list?
6 webdoctor 8:57 am Nov 20, 2006