Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Click-thru abuse or log spammer? |
6 | wilderness | 8:41 pm July 28, 2008 | ||||
| Snoop |
2 | wilderness | 11:45 am July 26, 2008 | ||||
| ODP/dmoz.org link checker pmoz bot does not read robots.txt |
17 | Umbra | 8:50 pm July 25, 2008 | ||||
| No IP IP of bot just a dot |
8 | farflappin | 3:21 pm July 24, 2008 | ||||
| betaBot |
2 | keyplyr | 1:00 am July 23, 2008 | ||||
| extrabot dot com |
4 | keyplyr | 7:15 pm July 21, 2008 | ||||
| Visbot returns With a proper UA and webmaster info page |
2 | jdMorgan | 7:40 pm July 19, 2008 | ||||
| Web Crawlers basics beginner need help! |
2 | PhoShzzle | 4:13 pm July 17, 2008 | ||||
| Oozbot/0.17 |
8 | incrediBILL | 11:26 pm July 15, 2008 | ||||
| Munax bots cloaking themselves and causing high server load |
16 | Asia_Expat | 7:49 pm July 15, 2008 | ||||
| Webbot/0.1 Russian bot |
3 | keyplyr | 11:29 pm July 14, 2008 | ||||
| MSN bot finds php robots.txt |
4 | phred | 8:26 am July 14, 2008 | ||||
| CatchBot |
4 | wilderness | 11:59 pm July 13, 2008 | ||||
| WebSense Hiding Behind Rotating User Agents Internet Security Company Tricks Site Security |
20 | Not_academic | 4:11 pm July 11, 2008 | ||||
| Google 66.249.84. |
2 | wilderness | 5:33 am July 11, 2008 | ||||
| Googlebot / image bot causing probs? sometimes hitting 404 pg, sometimes skips |
14 | Megaclinium | 12:19 am July 11, 2008 | ||||
| Wanadoo some scraping? |
3 | incrediBILL | 12:18 am July 11, 2008 | ||||
| is bot from 207.200.116 netscape? just 'AOL 9.0' listed in UA |
4 | Megaclinium | 10:31 pm July 10, 2008 | ||||
| Google Web Accelerator ... it's back! |
6 | Umbra | 5:04 pm July 8, 2008 | ||||
| Spider testing UAs? 92.48.126.nnn |
3 | Mokita | 7:06 am July 8, 2008 | ||||
| Mr.Carlito |
4 | idiotgirl | 1:36 pm July 7, 2008 | ||||
| Register Scolds AVG For Generating Fake Traffic As Link Malware Webmasters Complain AVG Debilitating Traffic Analytics[8] ( 1 2 3 4 5 6 7 8 ) |
219 | Samizdata | 7:49 am July 7, 2008 | ||||
| CyberPatrol SiteCat Webbot It crawls the internet looking for harmful sites |
5 | thetrasher | 4:31 am July 7, 2008 | ||||
| CFNetwork/330 |
3 | Receptional_Andy | 11:23 pm July 6, 2008 | ||||
| get 'head' is bots? noticed new gets with this notation |
5 | Megaclinium | 5:59 am July 6, 2008 |
