Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| true.net business internet or hosting? |
7 | incrediBILL | 7:18 pm May 17, 2013 | ||||
| above the law? ignores ROBOTS.TXT |
13 | lucy24 | 10:41 am May 17, 2013 | ||||
| Amazon AWS Hosts Bad Bots amazonaws.com[3] ( 1 2 3 ) |
88 | Pfui | 11:07 pm May 16, 2013 | ||||
| AcoonBot (Leaseweb Germany) 37.58.52.15 |
7 | Bewenched | 9:44 pm May 12, 2013 | ||||
| Cyberbunker |
9 | keyplyr | 7:00 am May 11, 2013 | ||||
| Firefox Browser just passing through |
3 | lucy24 | 6:50 pm May 8, 2013 | ||||
| Wotbox/2.0 is cheating from DSL hosting Contains unauthorized content |
2 | incrediBILL | 8:18 pm May 6, 2013 | ||||
| Server Farms - March 2013 Ongoing WMW server farm report[4] ( 1 2 3 4 ) |
92 | wilderness | 8:15 pm May 5, 2013 | ||||
| Don't stop fearing the webreaper |
6 | lucy24 | 7:53 pm May 4, 2013 | ||||
| Ipod, Ipad Iphone user agents Any bad conduct coming from such UAs |
3 | slipkid | 3:12 am May 3, 2013 | ||||
| British Library bot UK law now requires British Library to harvest UK web sites. |
3 | dstiles | 1:56 pm May 1, 2013 | ||||
| How Blocking Impacts Traffic and Other Web Services |
2 | incrediBILL | 8:02 pm Apr 30, 2013 | ||||
| stremorbot |
15 | lucy24 | 5:51 pm Apr 22, 2013 | ||||
| hosting company out of Japan 203.189.109.108 |
3 | Bewenched | 3:53 am Apr 19, 2013 | ||||
| who are piclab and what do they want with us? |
4 | lucy24 | 2:31 am Apr 19, 2013 | ||||
| checks.panopta.com User Agent Another site monitor |
16 | grandma_genie | 1:52 pm Apr 16, 2013 | ||||
| ip-web-crawler - stupidity never goes out of fashion |
8 | lucy24 | 8:54 pm Apr 10, 2013 | ||||
| ScreenerBot |
3 | Ocean10000 | 8:28 pm Apr 10, 2013 | ||||
| Fake BingBot 1000s of hits from a Slicehost IP[2] ( 1 2 ) |
31 | Gaia | 9:29 pm Apr 3, 2013 | ||||
| facebookexternalhit - Accessing /images/ now? |
7 | brokaddr | 9:36 pm Mar 28, 2013 | ||||
| Are there ANY legitimate visits from AWS? |
15 | 1script | 6:30 pm Mar 27, 2013 | ||||
| Detecting human users via user agent Not looking for 100% accuracy, close enough will do |
6 | bhonda | 2:27 am Mar 25, 2013 | ||||
| msresearch.us cloaked MSN spawn |
3 | Pfui | 3:25 am Mar 21, 2013 | ||||
| Static IP Services |
4 | wilderness | 1:43 pm Mar 18, 2013 | ||||
| Server Farms, Elron Technologies and more [6] ( 1 2 3 4 5 6 ) |
169 | wilderness | 12:24 am Mar 8, 2013 |
