Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Facebook? Search engine, what? |
11 | jake66 | 4:01 am Mar 12, 2010 | ||||
| Multiple sites showing similar crawl spikes |
4 | DiscoStu | 1:52 am Mar 11, 2010 | ||||
| 'Rippers 0' user agent from corporate networks UA looks like a ripper, but netork and behaviour do not |
11 | graeme_p | 6:44 am Mar 9, 2010 | ||||
| Speedy Spider New UA |
4 | keyplyr | 6:14 pm Mar 8, 2010 | ||||
| Shopwiki bot IPs |
6 | dstiles | 6:29 pm Mar 7, 2010 | ||||
| CSimpleSpider Robot |
3 | keyplyr | 1:31 am Mar 7, 2010 | ||||
| Wtabot |
2 | Pfui | 8:41 pm Mar 3, 2010 | ||||
| Host as IP Illegal access to sites without domain in URL |
5 | dstiles | 2:44 am Feb 28, 2010 | ||||
| Help identify attack tool with user agent: "Mozilla/5.0" Server exploit attacks from APNIC use this user agent |
8 | Wizcrafts | 1:01 am Feb 28, 2010 | ||||
| What is a web crawler? 2: |
2 | seedyplayer | 5:34 am Feb 23, 2010 | ||||
| Digsby IM Enables Web Crawlers Control of Your PC & Bandwidth Plura Processing and 80Legs to Leverage Digsby Network[3] ( 1 2 3 ) |
78 | incrediBILL | 1:58 pm Feb 21, 2010 | ||||
| Huge Trident UserAgent String trident user agent string |
15 | cyberdyne | 5:05 pm Feb 20, 2010 | ||||
| ShopWiki Really messed up; really messes things up. |
4 | Pfui | 12:06 am Feb 20, 2010 | ||||
| ua-bot |
2 | keyplyr | 7:03 am Feb 17, 2010 | ||||
| drone |
2 | Pfui | 5:09 pm Feb 16, 2010 | ||||
| comodo.com (no UA) |
13 | Pfui | 3:32 pm Feb 14, 2010 | ||||
| Ask.com cloaked bot |
5 | keyplyr | 9:16 am Feb 14, 2010 | ||||
| Braindead Scraper With Garbage UA Braindead Scrapers Garbage UA |
4 | jmccormac | 11:50 pm Feb 13, 2010 | ||||
| New Trend Micro Shenenigans changed HTTP_ACCEPT_LANGUAGE header |
3 | caribguy | 12:16 am Feb 13, 2010 | ||||
| Ask.com spider with blank user agent |
5 | Lain_se | 6:52 pm Feb 11, 2010 | ||||
| vanoppen.biz |
3 | keyplyr | 2:52 pm Feb 10, 2010 | ||||
| Cugillion Search Engine Robot New scraper; no robots.txt |
14 | Pfui | 5:42 pm Feb 4, 2010 | ||||
| NetinfoBot |
2 | keyplyr | 10:45 pm Jan 30, 2010 | ||||
| zomba-bot |
3 | keyplyr | 9:32 pm Jan 28, 2010 | ||||
| Microsoft MSN SocialStreams Bot |
6 | keyplyr | 5:02 am Jan 26, 2010 |
