Welcome to WebmasterWorld Guest from 34.228.115.216

Forum Moderated by: Ocean10000

Search Engine Spider and User Agent Identification


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Server Farms - July 2018
Continuing discussion of hosting company IP ranges[5] ( 1 2 3 4 5 )
127 keyplyr 4:52 pm Feb 12, 2019
Robot's User Agent String
What webmasters want
1 keyplyr 9:34 pm Oct 14, 2018
At Home with the Robots: 2017 edition
Critique of the Year's Active User Agents
15 lucy24 3:52 pm May 18, 2017
Blocking Methods
How to block access to your server
1 keyplyr 6:29 pm May 10, 2017
User Agents of Programming Libraries and Command Line Tools
Common user agents
14 incrediBILL 8:54 pm Apr 29, 2008
104.199.128.134
7 andrew_thom 2:19 am Feb 24, 2019
 
0 12:00 am Jan 1, 1970
Google polyfill
3 lucy24 2:15 am Feb 24, 2019
Skotina via TOR
After so many years, scrapers are still there...
3 blend27 11:49 pm Feb 13, 2019
Hits from 104.132.0.0/16 (Google) - human or bot?
7 SumGuy 2:29 am Feb 12, 2019
Google-speakr
2 phranque 6:21 am Feb 5, 2019
Seeing hits from 3.0.0.0/8 for first time
Is Amazon
8 SumGuy 2:41 am Feb 4, 2019
URLTester
3 lucy24 6:07 am Feb 3, 2019
Symantec from **.broadband.digiweb.ie
The range belongs to Symantec Corporation
7 blend27 10:09 pm Jan 28, 2019
Microsoft and 13.64.0.0/11
Legit bing-bots? BingLocalSearch?
14 SumGuy 7:04 pm Jan 21, 2019
Discordbot
3 keyplyr 2:40 am Jan 14, 2019
iCab/5.8.4
2 TorontoBoy 1:03 am Jan 13, 2019
Namerimi Nutch Spider
1 lucy24 8:41 pm Dec 27, 2018
7Siters
3 TorontoBoy 5:34 pm Dec 24, 2018
New Range for COCCOC bot?
4 blend27 7:34 pm Dec 15, 2018
facebookexternalhit
8 lucy24 2:54 pm Dec 10, 2018
Laserlikebot/0.1
2 TorontoBoy 4:55 am Dec 5, 2018
http.rb/3.3.0 (Mastodon/2.6.1;
1 TorontoBoy 1:56 am Dec 5, 2018
CowBot/0.4
1 TorontoBoy 12:25 am Dec 5, 2018
AutomaticWPTester
1 TorontoBoy 11:52 pm Dec 4, 2018

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members