Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  arste.info bot
2 Pfui 8:35 am Aug 30, 2009
  jc unknown var cloak.daily-2009-08-25-62-00.vars.referer
5 1EightT 10:22 pm Aug 26, 2009
  GingerCrawler/1.0
12 Mokita 4:18 pm Aug 23, 2009
  FunWebProducts, is it more than just a browser add-on?
4 JS_Harris 3:50 am Aug 23, 2009
  Is returning a blank page to known search crawlers acceptable?
Filtering search crawler clicks to affiliate links
3 JS_Harris 12:26 am Aug 23, 2009
  Google
Relative links
6 wilderness 7:11 am Aug 21, 2009
  Toata dragostea mea pentru diavola
All my love is for the devil
3 Ocean10000 4:34 pm Aug 20, 2009
  Windows 3.
3 wilderness 4:28 pm Aug 20, 2009
  Eurobot
2 keyplyr 4:25 pm Aug 20, 2009
  MS weirdness
5 keyplyr 3:12 pm Aug 19, 2009
  baypup/1.1 (Baypup; http://www.baypup.com/; jason@baypup.com)
3 GaryK 3:28 pm Aug 17, 2009
  Thunderstone
2 wilderness 6:23 am Aug 17, 2009
  Website Explorer/0.9.9.7
2 GaryK 11:51 pm Aug 16, 2009
  Sosospider
Another Rude Spider
3 Ocean10000 8:22 pm Aug 16, 2009
  What is User Agent "U"
found user agent or bot in our logfiles, dientified by U
8 Poolcenter 7:11 pm Aug 15, 2009
  atraxbot/0.2
16 keyplyr 6:46 pm Aug 11, 2009
  Google proxy junk
3 dstiles 7:04 pm Aug 10, 2009
  Rewrite ranges
12 wilderness 2:40 am Aug 10, 2009
  pigmej, wykop/0.3
18 Pfui 10:05 pm Aug 9, 2009
  http://www.amagit.com/
distributed search engine posing as a "new web" browser
3 GaryK 3:43 pm Aug 9, 2009
  Mozilla/5.0 (compatible; Firefox Addon; Windows XP 5.1)
2 GaryK 3:35 pm Aug 9, 2009
  southwebventures.com
2 keyplyr 2:55 pm Aug 7, 2009
  50.nu
4 Pfui 9:09 am Aug 7, 2009
  Colo's
9 wilderness 3:26 am Aug 7, 2009
  Dav
9 wilderness 1:50 pm Aug 6, 2009