Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| hubpages Bot-runner & UA |
3 | Pfui | 6:56 am Oct 15, 2011 | ||||
| Cutop |
7 | Pfui | 9:46 am Oct 9, 2011 | ||||
| suggybot née sgcrawlerbot |
5 | Pfui | 6:18 am Oct 9, 2011 | ||||
| Aboundex |
3 | Pfui | 7:57 pm Oct 8, 2011 | ||||
| What's the Best Way to Keep All Spiders/Bots Out? Only want a couple of pages crawled[2] ( 1 2 ) |
42 | inbound | 8:30 pm Oct 5, 2011 | ||||
| Amazon AWS gunning for Google? |
10 | Pfui | 6:41 pm Oct 3, 2011 | ||||
| Identifying Whitelist Bots Different approach |
12 | tangor | 6:08 pm Oct 2, 2011 | ||||
| MSN's many cloaked bots. Again. [6] ( 1 2 3 4 5 6 ) |
152 | Pfui | 12:40 am Oct 2, 2011 | ||||
| Google Keyword Suggestion |
2 | Pfui | 7:15 pm Oct 1, 2011 | ||||
| swish-e |
2 | Pfui | 4:20 am Oct 1, 2011 | ||||
| AndroidDownloadManger ...and other iterations |
4 | tangor | 1:53 am Oct 1, 2011 | ||||
| MSN bot hitting hard without a referrer 65.52.104.27 |
10 | SEOPTI | 12:00 am Sept 29, 2011 | ||||
| vBSEO User Agent Question about referer |
4 | grandma_genie | 6:11 pm Sept 28, 2011 | ||||
| Why I'm Somewhat MIA [2] ( 1 2 ) |
31 | incrediBILL | 5:56 pm Sept 28, 2011 | ||||
| FlipboardProxy At least 2 versions; 2 cases (2 much) |
3 | Pfui | 12:20 pm Sept 27, 2011 | ||||
| Kavende Iranian Crawler |
4 | g1smd | 2:33 pm Sept 25, 2011 | ||||
| today's head-scratcher |
8 | lucy24 | 2:05 am Sept 25, 2011 | ||||
| Gootkit auto-rooter scanner Malicious virus implanter from dynamic IPs |
6 | dstiles | 10:15 pm Sept 24, 2011 | ||||
| DIY Botnet Kit It gets worse! |
3 | dstiles | 4:58 pm Sept 24, 2011 | ||||
| amazonaws.com plays host to wide variety of bad bots Most recently seen: Gnomit[10] ( 1 2 3 4 5 6 7 8 9 10 ) |
278 | Pfui | 10:30 pm Sept 23, 2011 | ||||
| .htaccess file for blocking Scraper/Spam Bots Htaccess File to Help with blocking Scraper/Spam Bots |
5 | sandman22 | 7:59 pm Sept 22, 2011 | ||||
| Feedfetcher every 40 minutes for two days feedfetcher persistently looking for non existent item |
17 | revrob | 9:22 pm Sept 21, 2011 | ||||
| Grafix-is |
6 | keyplyr | 8:05 pm Sept 20, 2011 | ||||
| Level 3 redux |
6 | Pfui | 3:25 pm Sept 19, 2011 | ||||
| Resolving "urlresolver" Google IPs repeat no-robots runs |
30 | Pfui | 4:55 am Sept 19, 2011 |
