Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| TECOMAC-Crawler/0.4 Doing a very deep crawl |
5 | warry | 12:19 am Mar 12, 2003 | ||||
| 6 bots left... What's the safest identification method? |
8 | menyak | 8:36 pm Mar 11, 2003 | ||||
| "Phantom" bot help in identifying invisible bot |
4 | bobmark | 3:58 pm Mar 11, 2003 | ||||
| Fake bot spotted using Googlebot agent name |
12 | jcoronella | 2:59 pm Mar 11, 2003 | ||||
| Not see these googlebots before ! are they new googlebots |
5 | ncw164x | 12:29 pm Mar 11, 2003 | ||||
| Image Maps Do Spiders follow... |
2 | fashezee | 10:46 am Mar 11, 2003 | ||||
| rico/0.1 New Spider? |
2 | react | 3:24 pm Mar 10, 2003 | ||||
| Log Soam Bots |
4 | wilderness | 6:16 am Mar 10, 2003 | ||||
| eMail Harvester UA's Identification by incoming spam |
24 | weesnich | 11:24 pm Mar 9, 2003 | ||||
| Submit it spider |
4 | Skylo | 7:41 pm Mar 9, 2003 | ||||
| 'grub' uhh, what is he? |
4 | ga_ga | 3:56 pm Mar 9, 2003 | ||||
| Could this be a spider thousands of hits from a machine running W2K IE5 |
3 | nikar | 1:28 pm Mar 9, 2003 | ||||
| New User Agent visited my site |
2 | ned911 | 3:29 pm Mar 8, 2003 | ||||
| Fast PartnerSite Crawler Part Two - Ammending Prior Post of Same Title |
2 | pendanticist | 5:05 pm Mar 6, 2003 | ||||
| Spider and Flash Which spiders could now submit websites in Flash? |
2 | Hafsa | 3:21 am Mar 6, 2003 | ||||
| How do you block requests with no UA Would this work? |
10 | Kerrin | 7:53 pm Mar 5, 2003 | ||||
| HeinrichderMiragoRobot What's this? |
3 | Scooter24 | 5:55 pm Mar 5, 2003 | ||||
| Scooter-W3.1.2 (infoSpider URL check) Anybody seen this one? |
4 | volatilegx | 2:59 pm Mar 5, 2003 | ||||
| New member intro |
10 | acronym | 12:14 am Mar 5, 2003 | ||||
| Msn preview |
4 | wilderness | 8:47 pm Mar 4, 2003 | ||||
| Can your data base effect the deapth of the search PHP site with links to HTML pages |
3 | Travel_man | 11:34 am Mar 4, 2003 | ||||
| Mouse-House Spider Anyone know anything about this spider? |
3 | react | 8:46 pm Mar 3, 2003 | ||||
| Heavily Armed Browser okay... what/who is this, and should I be worried? |
2 | Mad_Max | 10:35 am Mar 3, 2003 | ||||
| FlickBot 2.0 RPT-HTTPClient/0.3-3 |
3 | frontpage | 9:28 pm Mar 2, 2003 | ||||
| are these ips websense? |
9 | leoo24 | 4:57 pm Mar 2, 2003 |
