Welcome to WebmasterWorld Guest from 3.229.122.219
Forum Moderated by: Ocean10000
Search Engine Spider and User Agent Identification
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Server Farms - July 2018 Continuing discussion of hosting company IP ranges[5] ( 1 2 3 4 5 ) |
134 | keyplyr | 3:24 am Nov 21, 2019 | ||||
| Robot's User Agent String What webmasters want |
1 | keyplyr | 9:34 pm Oct 14, 2018 | ||||
| At Home with the Robots: 2017 edition Critique of the Year's Active User Agents |
15 | lucy24 | 3:52 pm May 18, 2017 | ||||
| Blocking Methods How to block access to your server |
1 | keyplyr | 6:29 pm May 10, 2017 | ||||
| User Agents of Programming Libraries and Command Line Tools Common user agents |
14 | incrediBILL | 8:54 pm Apr 29, 2008 | ||||
| 0 | 12:00 am Jan 1, 1970 | ||||||
| woorank |
1 | lucy24 | 9:16 pm Dec 8, 2019 | ||||
| Corax |
3 | tangor | 6:02 pm Dec 7, 2019 | ||||
| Bot Mashing |
6 | rivsrush | 7:55 am Dec 7, 2019 | ||||
| Linespider |
9 | lucy24 | 11:16 am Nov 21, 2019 | ||||
| Any humans come from there? *.compute.hwclouds-dns.com |
4 | blend27 | 6:02 am Nov 17, 2019 | ||||
| ApiTool |
7 | Pfui | 2:26 am Nov 17, 2019 | ||||
| Python, Curl and Robots.txt |
17 | dstiles | 8:48 pm Nov 12, 2019 | ||||
| Google Adsense Preview |
1 | JorgeV | 9:04 am Oct 22, 2019 | ||||
| whisper |
14 | lucy24 | 10:00 am Oct 20, 2019 | ||||
| Return / continuation of msnbot search.msn.com/msnbot.html |
11 | dstiles | 9:58 am Oct 20, 2019 | ||||
| Google hiccup |
3 | wilderness | 9:33 am Oct 14, 2019 | ||||
| Google-Read-Aloud |
1 | lucy24 | 8:48 pm Oct 6, 2019 | ||||
| Hi Pages exploit UA |
15 | Pfui | 5:26 am Oct 5, 2019 | ||||
| URI: ./ ./ mnt/ custom/ ProductDefinition DVR remote code execution |
3 | Pfui | 5:35 pm Sep 27, 2019 | ||||
| Yisou again |
2 | lucy24 | 12:06 am Sep 27, 2019 | ||||
| GrumpyCrawler Microsoft range |
3 | Pfui | 6:02 pm Sep 24, 2019 | ||||
| Thinking of blocking 40.64.0.0/10 and 51.140.0.0/14 (MSFT) Seen some bot stuff recently |
2 | SumGuy | 5:53 am Sep 24, 2019 | ||||
| myAgent Microsoft range |
1 | Pfui | 11:45 am Sep 23, 2019 | ||||
| Mb2345Browser/9.0 I'm pretty sure this is actually a crawler, not a browser |
8 | notriddle | 2:51 am Sep 10, 2019 |
Join The Conversation
- Register For Free! - Become a Pro Member!
- See forum categories - Enter the Forum
Moderators and Top Contributors
- Moderator List | Top Contributors:This Week, This Month, Nov, Oct, Archive, Top 100 All Time, Top Voted Members
Hot Threads This Week
- Google Updates and SERP Changes - December 2019
- Google Updates and SERP Changes - November 2019
- December 2019 AdSense Earnings & Observations
- UK Search Engine, Mojeek Invests in New Servers
- RIPE NCC Runs Out of IPv4 Addresses: Calls For Faster Progress on IPv6
- Sundar Pichai Appointed Alphabet CEO
- .Org Registry Acquired By Private Equity Company
- Sir Tim Berners-Lee: "Contract for the Web" With Backing From Over 80 Organisations
- Google Spam & Hacked Machines
- Changed Website platform HTML to WordPress then Website Ranking Drop
Featured Threads
- Google Updates and SERP Changes - December 2019
- UK Search Engine, Mojeek Invests in New Servers
- Sundar Pichai Appointed Alphabet CEO
- RIPE NCC Runs Out of IPv4 Addresses: Calls For Faster Progress on IPv6
- Sir Tim Berners-Lee: "Contract for the Web" With Backing From Over 80 Organisations
- Twitter Adds 2FA With No Need For A Phone Number
- .Org Registry Acquired By Private Equity Company
- Return of CPU Vulnerability: ZombieLoad Attack
- Google Updates and SERP Changes - November 2019
- Google Rolls Out Search Console Speed Report