Welcome to WebmasterWorld Guest from 18.234.111.56
Forum Moderated by: Ocean10000
Search Engine Spider and User Agent Identification
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Server Farms - July 2018 Continuing discussion of hosting company IP ranges[5] ( 1 2 3 4 5 ) |
127 | keyplyr | 4:52 pm Feb 12, 2019 | ||||
| Robot's User Agent String What webmasters want |
1 | keyplyr | 9:34 pm Oct 14, 2018 | ||||
| At Home with the Robots: 2017 edition Critique of the Year's Active User Agents |
15 | lucy24 | 3:52 pm May 18, 2017 | ||||
| Blocking Methods How to block access to your server |
1 | keyplyr | 6:29 pm May 10, 2017 | ||||
| User Agents of Programming Libraries and Command Line Tools Common user agents |
14 | incrediBILL | 8:54 pm Apr 29, 2008 | ||||
| AHC/2.1 from UK Amazon 3.8. |
1 | not2easy | 3:43 pm Apr 20, 2019 | ||||
| possible new bing crawl range |
2 | lucy24 | 1:54 am Apr 12, 2019 | ||||
| Bots start looking for back.zip, backup.zip, etc. Get rid of those if you have them in root. |
3 | SumGuy | 12:32 pm Apr 10, 2019 | ||||
| Google-speakr |
6 | phranque | 9:23 pm Apr 5, 2019 | ||||
| User Agent is Microsoft Office something or other |
5 | SumGuy | 8:29 pm Apr 4, 2019 | ||||
| Google 34 |
2 | lucy24 | 1:48 am Apr 2, 2019 | ||||
| Facebook cortex |
8 | lucy24 | 1:21 am Apr 2, 2019 | ||||
| 35.244.0.0/14 - Google or Merit (U-Mich) ? |
4 | SumGuy | 6:23 pm Apr 1, 2019 | ||||
| Getting lots of hits from this Cloudsigma. what's it? |
3 | born2run | 5:58 am Mar 24, 2019 | ||||
| Apple applebot increasing activity Increasing hits to interior web pages |
4 | SumGuy | 4:32 am Mar 18, 2019 | ||||
| At Home with the Robots: 2019 edition |
3 | lucy24 | 3:52 am Mar 13, 2019 | ||||
| Moreover |
3 | lucy24 | 2:01 am Mar 12, 2019 | ||||
| newspaper |
4 | Pfui | 4:13 am Feb 27, 2019 | ||||
| Google polyfill |
8 | lucy24 | 3:34 am Feb 25, 2019 | ||||
| 104.199.128.134 |
7 | andrew_thom | 2:19 am Feb 24, 2019 | ||||
| Skotina via TOR After so many years, scrapers are still there... |
3 | blend27 | 11:49 pm Feb 13, 2019 | ||||
| Hits from 104.132.0.0/16 (Google) - human or bot? |
7 | SumGuy | 2:29 am Feb 12, 2019 | ||||
| Seeing hits from 3.0.0.0/8 for first time Is Amazon |
8 | SumGuy | 2:41 am Feb 4, 2019 | ||||
| URLTester |
3 | lucy24 | 6:07 am Feb 3, 2019 | ||||
| Symantec from **.broadband.digiweb.ie The range belongs to Symantec Corporation |
7 | blend27 | 10:09 pm Jan 28, 2019 |
Join The Conversation
- Register For Free! - Become a Pro Member!
- See forum categories - Enter the Forum
Moderators and Top Contributors
- Moderator List | Top Contributors:This Week, This Month, Mar, Feb, Archive, Top 100 All Time, Top Voted Members
Hot Threads This Week
- April 2019 AdSense Earnings & Observations
- Chitika Shutting Down Immediately
- Is Google Attempting To Force Site Owners Into AdWords?
- How to get more Twitter followers
- Google testing before: and after: commands to help find old pages
- Google AMP Web Results Link to Signed Exchanges
- Google Updates and SERP Changes - March 2019
- Should I devote time to Voice Search?
- Rating an Adsense account
- We are an old-school lot around here
Featured Threads
- Chitika Shutting Down Immediately
- Google AMP Web Results Link to Signed Exchanges
- How to get more Twitter followers
- Google testing before: and after: commands to help find old pages
- Google Updates and SERP Changes - April 2019
- Reports of Google Dropping Pages Out of Index... possibly a bug?
- So long, farewell, AdSense, goodbye
- Facebook Introduces "Why am I seeing this post?"
- Google Florida Update v2 March 12, 2019
- How to Prepare and Recover From Google Updates