Search Engine Spider and User Agent Identification |
| Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL | Forum Options: Reset Last Read Pointers |
PreModerated forum to id search engine spiders |
| | |
| new | Subject | Messages | Started by | Last Message |
 |
  Server Farms - May 2013[2] ( 1 2 )
Ongoing Hosting Data Center Discussion | 39 | incrediBILL | 6:54 pm on Jun 12, 2013 (gmt 0) |
|
|
Default User Agents of Programming Libraries and Command Line Tools
Resource page for common user agents | 15 | incrediBILL | 8:54 pm on Apr 29, 2008 (gmt 0) |
|
|
Quick primer on identifying bot activity.
And a how to guide to slow and stop scraping | 21 | Ocean10000 | 12:48 pm on Apr 4, 2008 (gmt 0) |
|
| good riddance to bad robots
| 3 | lucy24 | 6:25 pm on Jun 17, 2013 (gmt 0) |
|
| Request Protocol
| 12 | wilderness | 9:28 pm on Jun 13, 2013 (gmt 0) |
|
| Amazon AWS Hosts Bad Bots
Continuation Thread | 9 | incrediBILL | 2:05 am on Jun 11, 2013 (gmt 0) |
|
| Bots coming from Peg Tech
198.2.192.0/18 | 5 | lucy24 | 9:42 pm on Jun 10, 2013 (gmt 0) |
|
| Front Range & Pagesinvenotry
| 3 | Ken_S | 9:08 pm on Jun 7, 2013 (gmt 0) |
|
| Bad behavior from Microsoft IP
| 10 | keyplyr | 7:44 pm on Jun 6, 2013 (gmt 0) |
|
| Goodby old "friends" - Hello Watson
I.B.M. buys SoftLayer for 2bln | 2 | blend27 | 11:50 am on Jun 6, 2013 (gmt 0) |
|
| UNTRUSTED in Nokia User Agent[2] ( 1 2 )
| 44 | lucy24 | 11:41 am on Jun 4, 2013 (gmt 0) |
|
| Fortinet Technologies
| 3 | wilderness | 11:51 pm on Jun 3, 2013 (gmt 0) |
|
| Swapdrive Norton
| 2 | wilderness | 8:02 pm on Jun 3, 2013 (gmt 0) |
|
| htaccess and banning bots
code is not working for all bots | 18 | bthom62 | 7:39 pm on Jun 2, 2013 (gmt 0) |
|
| Strange Googlebot behavior
Grabbing same page over and over in one hit | 7 | grandma_genie | 4:57 pm on May 31, 2013 (gmt 0) |
|
| oBot robot where art thou?
| 5 | lucy24 | 9:16 pm on May 30, 2013 (gmt 0) |
|
| IANA Country Codes
No longer published as country codes? | 10 | dstiles | 9:22 pm on May 27, 2013 (gmt 0) |
|
| Needle
Ouch, here is a thread for you. | 11 | blend27 | 9:19 pm on May 27, 2013 (gmt 0) |
|
| Rackspace
198.61.128.0 - 198.61.255.255 | 7 | Bewenched | 9:18 pm on May 27, 2013 (gmt 0) |
|
| true.net business internet or hosting?
| 7 | incrediBILL | 7:18 pm on May 17, 2013 (gmt 0) |
|
| above the law? ignores ROBOTS.TXT
| 13 | lucy24 | 10:41 am on May 17, 2013 (gmt 0) |
|
| Amazon AWS Hosts Bad Bots[3] ( 1 2 3 )
amazonaws.com | 88 | Pfui | 11:07 pm on May 16, 2013 (gmt 0) |
|
| AcoonBot (Leaseweb Germany)
37.58.52.15 | 7 | Bewenched | 9:44 pm on May 12, 2013 (gmt 0) |
|
| Cyberbunker
| 9 | keyplyr | 7:00 am on May 11, 2013 (gmt 0) |
|
| Firefox Browser just passing through
| 3 | lucy24 | 6:50 pm on May 8, 2013 (gmt 0) |
| Displaying Page 1 of 208 | [ 1 ] 2 3 4 5 6 7 8 9 10 ... 208 Next >> |
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved
