Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Sap |
6 | wilderness | 11:48 pm Mar 27, 2008 | ||||
| User Agent: "WordPress/2.1.1" coming from Russia and UK Is this an RSS aggragator, or a scraper bot? |
6 | Wizcrafts | 2:32 am Mar 27, 2008 | ||||
| Readrun tag crawl showing up lately annoyance |
4 | mrjones | 9:52 am Mar 26, 2008 | ||||
| Slashers wondering about site slashers |
3 | Megaclinium | 6:22 am Mar 26, 2008 | ||||
| what is this bot? 72.244.103.zz |
3 | Megaclinium | 11:42 pm Mar 25, 2008 | ||||
| ConveraCrawler Is ConveraCrawler a malicious robot? |
17 | mattie | 1:05 pm Mar 25, 2008 | ||||
| Boitho ignoring robots.txt |
7 | Megaclinium | 8:17 pm Mar 24, 2008 | ||||
| Strange UA |
4 | wilderness | 1:14 am Mar 21, 2008 | ||||
| Pete-Spider Light |
2 | keyplyr | 11:24 pm Mar 20, 2008 | ||||
| TekSavvy |
8 | wilderness | 12:20 pm Mar 20, 2008 | ||||
| gnu-classpath Cute |
3 | wilderness | 9:27 pm Mar 19, 2008 | ||||
| zermelo |
3 | zerillos | 8:34 pm Mar 19, 2008 | ||||
| odd access log entry access log get form entry |
2 | esme | 10:03 pm Mar 18, 2008 | ||||
| anyone know this UA? all the spaces are +'s :? |
11 | wkitty42 | 4:07 pm Mar 18, 2008 | ||||
| agodar |
3 | Mokita | 10:28 pm Mar 17, 2008 | ||||
| User Agent "Firefox" |
8 | marodhum | 10:17 pm Mar 17, 2008 | ||||
| Ban spider by connection: close Header, (idea) I got an idea by ban spider by connection: close header |
15 | Eric | 9:04 am Mar 17, 2008 | ||||
| libwww-perl/5.53 archive.org |
10 | react | 7:32 am Mar 17, 2008 | ||||
| Quintura-Crw/0.1 Quintura-Crw/0.1 from IP 208.82.204.zz |
4 | Eric | 1:46 am Mar 17, 2008 | ||||
| New RSS Reader - Spider? Asa/1.0.1 |
2 | Ocean10000 | 1:07 am Mar 17, 2008 | ||||
| Jeanie 2008 bot No robots request, fell straight into spider trap |
2 | Receptional_Andy | 9:57 pm Mar 16, 2008 | ||||
| Yahoo Slurp China htaccess block Slurp China htaccess block |
13 | cyberdyne | 9:02 am Mar 8, 2008 | ||||
| strange assignment |
2 | wilderness | 10:47 am Mar 4, 2008 | ||||
| LTI/LemurProject another Nutch pest |
8 | keyplyr | 6:10 am Mar 3, 2008 | ||||
| T-h-u-n-d-e-r-s-t-o-n-e |
3 | wilderness | 6:20 pm Feb 29, 2008 |
