Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| bixolabs bot broken 1433 loads of the same URL over 21 hours |
6 | rowan194 | 5:04 am Aug 15, 2010 | ||||
| How to block all robots except google, yahoo, msn, and ask |
7 | vphoner | 7:58 pm Aug 14, 2010 | ||||
| Mozilla 5.0/Metal will never die |
2 | Dijkgraaf | 7:51 pm Aug 14, 2010 | ||||
| SemtoBot/0.1 |
2 | keyplyr | 3:32 pm Aug 12, 2010 | ||||
| Opera/9.80 (Windows NT 5.1; U; ru) Presto/2.6.30 Version/10.60 Russian scraper |
2 | Dijkgraaf | 2:03 am Aug 12, 2010 | ||||
| bitmagicbot |
10 | Pfui | 5:06 pm Aug 10, 2010 | ||||
| Why the obsession with Favicon? |
8 | Mokita | 12:31 pm Aug 10, 2010 | ||||
| Python-httplib2 |
2 | yaashul | 9:55 pm Aug 9, 2010 | ||||
| carrier-enabler |
3 | Pfui | 4:42 pm Aug 8, 2010 | ||||
| Yandex bots - incorrect rDNS? |
7 | dstiles | 9:40 pm Aug 6, 2010 | ||||
| How to Identify bad bots from legitimate users Need help on hints and tips that give away a bad bot |
11 | liam_85 | 3:43 pm Aug 6, 2010 | ||||
| Google%20Quick%20Search%20Box |
4 | Pfui | 12:29 pm Aug 4, 2010 | ||||
| CustomUserAgent Apple surprise? |
12 | Pfui | 6:16 pm Aug 1, 2010 | ||||
| Teleport Ultra |
3 | Pfui | 11:14 pm July 31, 2010 | ||||
| ARIN Whois |
11 | wilderness | 1:24 pm July 31, 2010 | ||||
| How to trap bots? That crawls disallowed pages. |
25 | Sunnz | 2:16 pm July 26, 2010 | ||||
| Botnet Attempting Zen Cart Attack Blank UAs Attacking |
13 | incrediBILL | 3:08 am July 24, 2010 | ||||
| wtv bot |
2 | Pfui | 3:40 am July 23, 2010 | ||||
| categorizers.bdbrandprotect.com referer bdbrandprotect http referer |
3 | a24061 | 8:51 pm July 21, 2010 | ||||
| acquia |
7 | Pfui | 8:18 pm July 19, 2010 | ||||
| panscient with own IP range |
4 | Staffa | 5:48 pm July 19, 2010 | ||||
| Interesting site ripper/bot like critter Try blocking this one (other than IP...) |
3 | tangor | 6:47 am July 18, 2010 | ||||
| Yahoo Pipes 1.0 as UA Seems like RSS Bot? |
2 | blend27 | 8:44 pm July 16, 2010 | ||||
| KiwiStatus/0.3 New Zealand Search |
2 | Dijkgraaf | 8:25 pm July 16, 2010 | ||||
| Gentoo UA - to block or not to block |
2 | zerillos | 7:22 pm July 15, 2010 |
