Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Slurp/2.0-Kiteweekly? Kiteweekly? |
6 | grnidone | 6:19 pm Oct 9, 2001 | ||||
| Mozilla/3.Mozilla/2.01 (Win95; I) from nucleus.com |
6 | idiotgirl | 9:18 am Oct 9, 2001 | ||||
| Mozilla/3.0 NAVIO (dtv0) Verio - Best.com - and Ravenous |
2 | bobriggs | 6:16 am Oct 6, 2001 | ||||
| Sources of SE IP numbers Anyone know of a frequently updated list of SE IP numbers? |
2 | Wallace | 9:25 pm Oct 5, 2001 | ||||
| compatible;+Indy+Library 211.101.236.93 |
3 | dolphin | 11:33 pm Oct 4, 2001 | ||||
| knowledge/0.5 Anyone heard of bot called "knowledge/0.5"? |
4 | LeBain | 9:10 pm Oct 3, 2001 | ||||
| InfoNaviRobot(F107) InfoNaviRobot(F107) |
4 | bot_watcher | 5:05 pm Oct 3, 2001 | ||||
| larbin2.1.1 - 137.203.128.34 137.203.128.34 |
15 | dolphin | 5:26 pm Oct 2, 2001 | ||||
| tracerlock spidering 2 pages every 15 minutes |
4 | bobriggs | 12:27 am Oct 1, 2001 | ||||
| Spider Language Do spiders have a specific language associated to them? |
4 | sweeties | 4:13 pm Sept 28, 2001 | ||||
| Does anyone know of where I can get a list of known spiders? |
7 | IanTurner | 6:14 pm Sept 26, 2001 | ||||
| http://www.almaden.ibm.com/cs/crawler/ |
4 | littleman | 12:27 pm Sept 26, 2001 | ||||
| GulperBot Open GRiD Project |
6 | bobriggs | 4:47 am Sept 25, 2001 | ||||
| About/0.1libwww-perl/5.47 |
4 | Marcia | 4:08 pm Sept 24, 2001 | ||||
| moget/2.1 (moget@goo.ne.jp) |
2 | littleman | 5:07 pm Sept 21, 2001 | ||||
| inktomi2-lee.server.ntl.com is this Hostname Inktomi? |
4 | volatilegx | 3:44 pm Sept 20, 2001 | ||||
| WebGather 3.0 61.139.65.222 |
5 | volatilegx | 8:16 am Sept 19, 2001 | ||||
| OneDisc.com What is this referrer? |
6 | grnidone | 3:00 am Sept 18, 2001 | ||||
| Mystery Spider 2001 |
5 | Brett_Tabke | 4:30 pm Sept 14, 2001 | ||||
| Internetseer |
6 | toolman | 4:25 pm Sept 14, 2001 | ||||
| what's pizilla? |
3 | proxy | 4:12 pm Sept 14, 2001 | ||||
| Danish spider? Marvin v0.5 - 130.225.20.4 |
4 | Will | 3:57 pm Sept 10, 2001 | ||||
| What kind of spider is this? WebCopier+v.2.2, ip - 24.88.11.3 |
2 | stanleytk | 12:52 pm Sept 10, 2001 | ||||
| UA Lifeboat followed by Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 ). |
2 | bobriggs | 6:09 pm Sept 8, 2001 | ||||
| wap spider or is this a browser??? |
3 | mack | 2:48 pm Sept 8, 2001 |
