Welcome to WebmasterWorld Guest from 54.224.44.168
Forum Moderated by: Ocean10000 & incrediBILL & keyplyr
Search Engine Spider and User Agent Identification
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Server Farms - Jan 2018 Continuing discussion of hosting company IP ranges[2] ( 1 2 ) |
54 | keyplyr | 8:13 pm Apr 25, 2018 | ||||
| At Home with the Robots: 2017 edition Critique of the Year's Active User Agents |
15 | lucy24 | 3:52 pm May 18, 2017 | ||||
| Blocking Methods How to block access to your server |
1 | keyplyr | 6:29 pm May 10, 2017 | ||||
| Default User Agents of Programming Libraries and Command Line Tools Resource page for common user agents |
15 | incrediBILL | 8:54 pm Apr 29, 2008 | ||||
| NYU Internet Census imposter |
7 | SumGuy | 11:59 pm Apr 26, 2018 | ||||
| 0 | 12:00 am Jan 1, 1970 | ||||||
| DownloaderChrome |
9 | lucy24 | 11:34 pm Apr 26, 2018 | ||||
| IP address range of ccbot user agent? |
4 | born2run | 10:53 pm Apr 26, 2018 | ||||
| Leikibot |
7 | keyplyr | 6:52 am Apr 25, 2018 | ||||
| Prudent to block empty user agent visitors? |
26 | born2run | 3:11 am Apr 25, 2018 | ||||
| Buck |
7 | keyplyr | 1:55 am Apr 25, 2018 | ||||
| Most of Your Traffic is Not Human [3] ( 1 2 3 ) |
69 | keyplyr | 1:20 pm Apr 24, 2018 | ||||
| VenusCrawler |
4 | keyplyr | 9:16 pm Apr 23, 2018 | ||||
| SurveyBot the new improved UA |
2 | lucy24 | 10:30 pm Apr 22, 2018 | ||||
| BacklinkCrawler |
2 | lucy24 | 9:28 pm Apr 22, 2018 | ||||
| another typo bites the dust |
2 | lucy24 | 8:57 pm Apr 22, 2018 | ||||
| The Knowledge AI |
5 | keyplyr | 1:16 am Apr 21, 2018 | ||||
| Yatke |
1 | keyplyr | 8:02 pm Apr 20, 2018 | ||||
| Kudelski Security Innovation Port Scanner |
5 | blend27 | 2:19 am Apr 18, 2018 | ||||
| MauiBot |
13 | keyplyr | 10:49 pm Apr 17, 2018 | ||||
| CORS bot |
1 | keyplyr | 10:59 pm Apr 16, 2018 | ||||
| EU Rejects Robots 'Electronic Persons' Status |
9 | keyplyr | 11:22 pm Apr 14, 2018 | ||||
| IndeedBot |
3 | TorontoBoy | 6:54 am Apr 13, 2018 | ||||
| ltx71 |
2 | keyplyr | 4:00 am Apr 12, 2018 | ||||
| Whither Semalt and the like? reducing list of active crawlers like semalt in htaccess |
9 | Martin_Potter | 7:54 pm Apr 10, 2018 |
Join The Conversation
- Register For Free! - Become a Pro Member!
- See forum categories - Enter the Forum
Moderators and Top Contributors
- Moderator List | Top Contributors:This Week, This Month, Mar, Feb, Archive, Top 100 All Time, Top Voted Members
Hot Threads This Week
- Google Updates and SERP Changes - April 2018
- April 2018 AdSense Earnings & Observations
- Facebook Goes for World-Wide ToS Changes
- Google AdSense: Expands "Funding Choices" To Replace Lost Revenue From Ad Blocking
- Google Updates PageRank Patent
- Chrome 66 for Windows, Mac and Linux Rolls Out
- What does EU GDPR Mean for Adsense?
- Google Updates and SERP Changes - March 2018
- Alphabet Q1 Revenues $31.1 Billion, up 26pct Year on Year
- What Makes Good Web Content Today
Featured Threads
- What does EU GDPR Mean for Adsense?
- Negative SEO: Apparently, It's Not so Easy to Poison a Website's Link Power
- New API Login Standards: WebAuthn and CTAP Published
- Google Updates PageRank Patent
- Alphabet Q1 Revenues $31.1 Billion, up 26pct Year on Year
- Facebook Goes for World-Wide ToS Changes
- Chrome 66 for Windows, Mac and Linux Rolls Out
- Google AdSense: Expands "Funding Choices" To Replace Lost Revenue From Ad Blocking
- What Makes Good Web Content Today
- Mozilla: Internet Health Report 2018