Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Hits from bigip1-snat.sv.av.com ( 209.73.164.50) Altavista new bot ? |
4 | bfleisch | 11:38 pm June 25, 2001 | ||||
| Heads up (as toolman would say) |
3 | Froggyman | 6:32 pm June 25, 2001 | ||||
| Is this an Alexa spider? |
3 | toolman | 4:08 pm June 25, 2001 | ||||
| Disney.com Spidering lwp-trivial/1.28 |
2 | Froggyman | 1:09 pm June 25, 2001 | ||||
| Kjjwro |
3 | jimmykav | 12:48 pm June 25, 2001 | ||||
| tivra spider |
2 | roryfrost | 12:29 am June 25, 2001 | ||||
| Mercator v1-2jg |
3 | startup | 5:17 am June 23, 2001 | ||||
| Spiders and counter stats |
3 | Outtatheblue | 5:05 pm June 22, 2001 | ||||
| Atom Indexer v1.1a 217.11.167.34 |
6 | skirril | 3:43 pm June 19, 2001 | ||||
| This is new... Ink UK? |
2 | toolman | 9:54 am June 19, 2001 | ||||
| Anyone know who this is? 209.67.206.133 |
3 | toolman | 8:16 am June 17, 2001 | ||||
| MSN what is the useragent |
2 | zipup | 9:02 pm June 15, 2001 | ||||
| List your banned IP's / domains and why |
7 | icehousedesigns | 1:06 pm June 14, 2001 | ||||
| Teoma Spider IP list where can I find one? |
5 | Travoli | 1:04 pm June 14, 2001 | ||||
| Crescent Internet ToolPak HTTP OLE Control v.1.0 Should I ban it in my robots.txt file? |
2 | merlin78 | 5:26 am June 14, 2001 | ||||
| Inktomi hitting pages over and over what the heck? |
5 | Travoli | 1:04 pm June 13, 2001 | ||||
| AOL UAbut different IP |
4 | toolman | 8:08 pm June 12, 2001 | ||||
| Site friendly spiders : the list |
4 | Brett_Tabke | 7:56 pm June 12, 2001 | ||||
| A new UA to me Fast |
2 | startup | 7:54 pm June 11, 2001 | ||||
| robot names - please help |
5 | Ade | 6:37 pm June 11, 2001 | ||||
| Spider Spammer Spammin' the UA |
3 | awoyo | 12:47 pm June 11, 2001 | ||||
| Anyone know what this is? larbin_2.1.0 larbin2.1.0@somewhere.com |
3 | icehousedesigns | 6:06 pm June 7, 2001 | ||||
| New 404 test from DIIbot |
2 | jimmykav | 6:04 pm June 7, 2001 | ||||
| MSIEcrawler For those of you that didn't know... |
3 | icehousedesigns | 3:35 pm June 7, 2001 | ||||
| 2 | Froggyman | 10:24 am June 7, 2001 |
