Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Scraper Test Drive Open Discussion of Scraper Tools and Success Rates |
3 | incrediBILL | 3:41 am Aug 31, 2013 | ||||
| First bot on IPv6 for me |
7 | bhukkel | 8:11 am Aug 28, 2013 | ||||
| Brand Affinity Tech |
6 | wilderness | 1:22 pm Aug 26, 2013 | ||||
| Did delegated-arin-latest vanish on 8.13.13? It seems to have disappeared from their server |
9 | incrediBILL | 4:54 am Aug 24, 2013 | ||||
| the plainclothes bingbot |
4 | lucy24 | 1:59 pm Aug 23, 2013 | ||||
| UP.Browser |
13 | lucy24 | 7:57 am Aug 14, 2013 | ||||
| App3leWebKit |
5 | lucy24 | 10:58 am Aug 13, 2013 | ||||
| IP addresses belonging to "your-server.de" How to keep certain spammers at bay |
3 | yaimapitu | 12:22 am Aug 11, 2013 | ||||
| .htaccess BadBot Blocker |
19 | JD_Toims | 4:48 am Aug 7, 2013 | ||||
| Synapse user-agent Mozilla/4.0 (compatible; Synapse)[2] ( 1 2 ) |
35 | dstiles | 7:00 pm Aug 6, 2013 | ||||
| Portlane |
9 | lucy24 | 7:45 am Aug 4, 2013 | ||||
| PromptCloud bot Web Extraction, crawling and scraping service |
4 | AnnieRogers | 8:19 pm Aug 1, 2013 | ||||
| Evernote |
2 | wilderness | 3:23 am July 30, 2013 | ||||
| mozilla position of "Mozilla" in UA string |
4 | lucy24 | 6:50 pm July 24, 2013 | ||||
| Baidu Hong Kong |
2 | lucy24 | 6:10 pm July 22, 2013 | ||||
| htaccess and banning bots code is not working for all bots |
25 | bthom62 | 4:40 pm July 5, 2013 | ||||
| Merchant Centric Bot |
3 | Ken_S | 3:40 am June 29, 2013 | ||||
| Possible bot |
4 | Readie | 9:50 pm June 26, 2013 | ||||
| MSNBot posing as Internet Explorer 7 and not reading robots.txt? |
11 | robzilla | 2:01 am June 26, 2013 | ||||
| June 2013 - Blocking Bad Spiders / Bots / User Agents |
26 | Imaster | 2:15 am June 25, 2013 | ||||
| ixquick, duckduckgo, startpage IP ranges of these search engines |
5 | motorhaven | 10:40 pm June 24, 2013 | ||||
| Bad behavior from Microsoft IP |
11 | keyplyr | 4:06 pm June 24, 2013 | ||||
| Bots coming from Peg Tech 198.2.192.0/18 |
6 | lucy24 | 8:53 pm June 22, 2013 | ||||
| good riddance to bad robots |
3 | lucy24 | 6:25 pm June 17, 2013 | ||||
| Request Protocol |
12 | wilderness | 9:28 pm June 13, 2013 |
