Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| My Spider report showing bots getting error and leaving Unknown error is keeping bots out of my site. |
12 | juniperwasting | 9:18 pm Mar 18, 2003 | ||||
| could "netscape 5" be a spider? |
4 | BillM | 7:02 am Mar 18, 2003 | ||||
| How to Ban Bots which come without http_user_agent info? It's real problem, I don't know how to set it up with rewrite mod |
11 | Chris_1977 | 2:52 am Mar 18, 2003 | ||||
| aol.com crawler = googlebot.com? |
9 | damiengazier | 1:42 am Mar 18, 2003 | ||||
| Lite Bot 0616B Ver numbers decreasing? |
3 | MarieC | 6:54 pm Mar 17, 2003 | ||||
| 216.191.20.26 |
3 | Brett_Tabke | 6:23 pm Mar 17, 2003 | ||||
| lachesis spider from Japan (Nara province) lachesis spider from Japan (Nara province) |
5 | misslulu | 2:10 pm Mar 17, 2003 | ||||
| PlantyNet_WebRobot_V1.9 Has anyone seen this? |
2 | MarieC | 12:26 pm Mar 16, 2003 | ||||
| obot spidered by 5 day old site |
3 | coffeelover | 9:11 am Mar 16, 2003 | ||||
| T-h-u-n-d-e-r-s-t-o-n-e Anyone know about this one? |
11 | pendanticist | 5:14 am Mar 16, 2003 | ||||
| 4 | StanTheMan | 7:28 pm Mar 15, 2003 | |||||
| Anyone know about UA: Datum/0.1 coming from IPs with a web server |
2 | acronym | 6:40 pm Mar 15, 2003 | ||||
| k2spider new UA coming out of Glaxo? |
2 | acronym | 7:15 am Mar 15, 2003 | ||||
| psbot/0.1 Anyone else getting hit by it? picsearch bot |
6 | Alphawolf | 2:17 am Mar 15, 2003 | ||||
| FAST-WebCrawler/3.7/FirstPage What does it do? |
5 | oLeon | 12:20 am Mar 15, 2003 | ||||
| Spider From China : Dynegy-communication SPIDER FROM CHINA : Zeus 2.6 |
2 | misslulu | 6:38 pm Mar 14, 2003 | ||||
| file vs directory better optimization |
2 | tthach2 | 5:41 am Mar 14, 2003 | ||||
| Turnititbot google spam reporting.....Nah |
5 | eljefe3 | 2:59 am Mar 14, 2003 | ||||
| lachesis spider form Japan (Nara province) lachesis spider form Japan (Nara province) |
4 | misslulu | 12:58 am Mar 14, 2003 | ||||
| Php/4.0.6 Double Calls for pages, with brief pause in between requests. |
3 | pendanticist | 6:20 am Mar 13, 2003 | ||||
| WebClippings old thread |
2 | wilderness | 5:05 am Mar 13, 2003 | ||||
| UA of major Search Engines made list of major UA serach engines |
3 | StopSpam | 7:50 pm Mar 12, 2003 | ||||
| Naverbot seems to have changed Possibel variant Taking big bites Now |
5 | carfac | 7:11 pm Mar 12, 2003 | ||||
| ctechld.com log spamming? |
5 | jazzguy | 2:42 am Mar 12, 2003 | ||||
| Kapere New(ish) site grabber? |
2 | ritch_b | 1:18 am Mar 12, 2003 |
