Search Engine Spider and User Agent Identification |
| Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL | Forum Options: Reset Last Read Pointers |
PreModerated forum to id search engine spiders |
| | |
| new | Subject | Messages | Started by | Last Message |
| |
  Server Farms - May 2013
Ongoing Hosting Data Center Discussion |
18 | incrediBILL | 7:08 pm on May 19, 2013 (gmt 0) |
| |
Default User Agents of Programming Libraries and Command Line Tools
Resource page for common user agents |
15 | incrediBILL | 8:54 pm on Apr 29, 2008 (gmt 0) |
| |
Quick primer on identifying bot activity.
And a how to guide to slow and stop scraping |
21 | Ocean10000 | 12:48 pm on Apr 4, 2008 (gmt 0) |
| | Request Protocol
|
7 | wilderness | 2:15 pm on May 18, 2013 (gmt 0) |
| | true.net business internet or hosting?
|
7 | incrediBILL | 7:18 pm on May 17, 2013 (gmt 0) |
| | above the law? ignores ROBOTS.TXT
|
13 | lucy24 | 10:41 am on May 17, 2013 (gmt 0) |
| | Amazon AWS Hosts Bad Bots
Continuation Thread |
1 | incrediBILL | 11:16 pm on May 16, 2013 (gmt 0) |
| | Amazon AWS Hosts Bad Bots[3] ( 1 2 3 )
amazonaws.com |
88 | Pfui | 11:07 pm on May 16, 2013 (gmt 0) |
| | Bad behavior from Microsoft IP
|
9 | keyplyr | 8:55 pm on May 16, 2013 (gmt 0) |
| | UNTRUSTED in Nokia User Agent[2] ( 1 2 )
|
42 | lucy24 | 9:04 pm on May 13, 2013 (gmt 0) |
| | AcoonBot (Leaseweb Germany)
37.58.52.15 |
7 | Bewenched | 9:44 pm on May 12, 2013 (gmt 0) |
| | Rackspace
198.61.128.0 - 198.61.255.255 |
3 | Bewenched | 8:04 pm on May 11, 2013 (gmt 0) |
| | Cyberbunker
|
9 | keyplyr | 7:00 am on May 11, 2013 (gmt 0) |
| | Firefox Browser just passing through
|
3 | lucy24 | 6:50 pm on May 8, 2013 (gmt 0) |
| | Wotbox/2.0 is cheating from DSL hosting
Contains unauthorized content |
2 | incrediBILL | 8:18 pm on May 6, 2013 (gmt 0) |
| | Server Farms - March 2013[4] ( 1 2 3 4 )
Ongoing WMW server farm report |
92 | wilderness | 8:15 pm on May 5, 2013 (gmt 0) |
| | Don't stop fearing the webreaper
|
6 | lucy24 | 7:53 pm on May 4, 2013 (gmt 0) |
| | Ipod, Ipad Iphone user agents
Any bad conduct coming from such UAs |
3 | slipkid | 3:12 am on May 3, 2013 (gmt 0) |
| | British Library bot
UK law now requires British Library to harvest UK web sites. |
3 | dstiles | 1:56 pm on May 1, 2013 (gmt 0) |
| | How Blocking Impacts Traffic and Other Web Services
|
2 | incrediBILL | 8:02 pm on Apr 30, 2013 (gmt 0) |
| | same old same old YisouSpider
Yisou back from the dead? |
2 | lucy24 | 7:49 pm on Apr 30, 2013 (gmt 0) |
| | Needle
Ouch, here is a thread for you. |
7 | blend27 | 4:51 am on Apr 23, 2013 (gmt 0) |
| | stremorbot
|
15 | lucy24 | 5:51 pm on Apr 22, 2013 (gmt 0) |
| | hosting company out of Japan
203.189.109.108 |
3 | Bewenched | 3:53 am on Apr 19, 2013 (gmt 0) |
| | who are piclab and what do they want with us?
|
4 | lucy24 | 2:31 am on Apr 19, 2013 (gmt 0) |
| Displaying Page 1 of 208 | [ 1 ] 2 3 4 5 6 7 8 9 10 ... 208 Next >> |
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved
