Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Temporary Absence Update |
8 | incrediBILL | 1:57 am Mar 6, 2013 | ||||
| Revised Spider IPs and Other Details At Home with the Robots: 2013 edition |
6 | lucy24 | 5:08 pm Mar 3, 2013 | ||||
| Facebook |
2 | topr8 | 8:20 am Feb 27, 2013 | ||||
| Softlayer |
13 | Ken_S | 9:43 pm Feb 21, 2013 | ||||
| Pinterest Domain Verifier |
6 | keyplyr | 11:50 pm Feb 20, 2013 | ||||
| Blekko how to ban |
5 | viggen | 12:41 am Feb 17, 2013 | ||||
| msnbot-media What's a nice robot like you doing in a place like this?[2] ( 1 2 ) |
58 | lucy24 | 4:25 am Feb 12, 2013 | ||||
| -very- much the same robot |
10 | lucy24 | 10:48 pm Feb 4, 2013 | ||||
| U of Cal Santa Barbara |
9 | wilderness | 11:19 am Feb 4, 2013 | ||||
| BingBot Fake |
19 | Ken_S | 3:45 am Feb 4, 2013 | ||||
| Yandex gets fatter |
13 | lucy24 | 9:57 pm Feb 1, 2013 | ||||
| the latest impenetrable disguise [2] ( 1 2 ) |
40 | lucy24 | 3:47 am Jan 30, 2013 | ||||
| goo goo GET GET |
8 | lucy24 | 7:50 am Jan 27, 2013 | ||||
| Filtering Out Really Hard To Find Bad Bots |
26 | incrediBILL | 2:16 am Jan 24, 2013 | ||||
| ZTE-Z431/1.4.0 NetFront/4.2 QTV5.1 User-Agent Profile/MIDP-2.1 Configuration/CLDC-1.1 |
4 | not2easy | 9:24 pm Jan 23, 2013 | ||||
| WBSearchBot misbehaving |
12 | incrediBILL | 12:11 am Jan 19, 2013 | ||||
| and another gets on. |
5 | lucy24 | 1:25 pm Jan 18, 2013 | ||||
| Lots of interesting user agents |
4 | Bewenched | 7:54 pm Jan 15, 2013 | ||||
| Baidu's IPs - Which Are Legit? |
12 | incrediBILL | 7:21 pm Jan 13, 2013 | ||||
| LSSRocketCrawler/1.0 LightspeedSystems 69.84.207.246 |
3 | Bewenched | 5:57 pm Jan 13, 2013 | ||||
| sitemap.xml requests |
3 | wilderness | 12:38 am Jan 13, 2013 | ||||
| Tiscali Communicator (scraper) on SteepHost (Ukraine) |
3 | not2easy | 9:22 pm Jan 12, 2013 | ||||
| Could someone spare a set of Anti-AWS (and other offenders) CSF rules? |
2 | 1script | 8:45 am Jan 12, 2013 | ||||
| Are these considered as bad bots ? |
23 | pkKumar | 3:00 am Jan 11, 2013 | ||||
| msn bot has gone crazy |
15 | bwnbwn | 10:52 pm Jan 9, 2013 |
