Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  ua: Hello
Now this one is nasty
4 FineWare 12:05 pm June 21, 2003
  Automatic identification/blocking
2 ephka 5:40 pm June 20, 2003
  InternetSeer.com
should i block this
4 sajjid 12:13 am June 20, 2003
  False agent-name and False IP
A real pain in the.......
15 sanuk 11:00 am June 19, 2003
  pleasdo as user agent?
any ideas?
2 Clark 9:34 pm June 18, 2003
  Canadian Crawler Hammered my site
Bell Nexxia?
5 kewlbeezer 10:00 pm June 17, 2003
  interseer or InterSeer
using NC and OR is ok?
4 sanuk 8:57 pm June 17, 2003
 
13 DavidT 8:45 pm June 17, 2003
  Checkbot/1.71
New bot? I couldn't find it in WW site search
9 berli 1:37 am June 17, 2003
  Another New Crawler - kljwdw3gfdrfdsfdsdfd
Looks like someone was lazy when they made this name...
3 kewlbeezer 10:29 pm June 16, 2003
  trapping spiders - Google ban result?
making a spider trap
8 sanuk 10:27 pm June 16, 2003
  Missouri College Browse
4 ned911 10:20 pm June 16, 2003
  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3a) Gecko/20021212
Is this Alta Vista?
6 GaryK 10:35 am June 15, 2003
  what's this: "GET /watch-info HTTP/1.0" 403
6 meinereiner 10:29 am June 15, 2003
  Anyone Know Which Spider this is?
Got hit hard by them this morning
16 cmjohnston 10:05 am June 14, 2003
  Singingfish extractor not obeying robots.txt
sound files hosted at different domain than website
3 Finder 12:51 am June 14, 2003
  YahooBB220048191035.bbtec.net
whats yahoo doin on my site lol
3 penfold25 5:11 pm June 13, 2003
  toCrawl/UrlDispatcher
another one
8 sanuk 11:55 am June 13, 2003
  Two CJ spiders
Two CJ Spiders
3 yogsco 5:06 am June 13, 2003
  Searchalot
6 wilderness 5:21 pm June 12, 2003
  Genie
What is this?
6 Oaf357 3:53 am June 12, 2003
  Vivante
ain't coming to *my* lifestyle, I guarantee.
2 SeanL 3:23 am June 12, 2003
  NY Internet Srvcs
anyone?
4 bull 3:01 am June 12, 2003
  Web Link Validator
a new spidering bot?
5 sanuk 11:40 pm June 11, 2003
  213.52.206.98
who is this?
3 Netizen 1:58 pm June 11, 2003