Welcome to WebmasterWorld Guest from 3.80.4.76

Forum Moderated by: Ocean10000

Search Engine Spider and User Agent Identification


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Server Farms - July 2018
Continuing discussion of hosting company IP ranges[5] ( 1 2 3 4 5 )
132 keyplyr 3:03 pm Jun 13, 2019
Robot's User Agent String
What webmasters want
1 keyplyr 9:34 pm Oct 14, 2018
At Home with the Robots: 2017 edition
Critique of the Year's Active User Agents
15 lucy24 3:52 pm May 18, 2017
Blocking Methods
How to block access to your server
1 keyplyr 6:29 pm May 10, 2017
User Agents of Programming Libraries and Command Line Tools
Common user agents
14 incrediBILL 8:54 pm Apr 29, 2008
myAgent
Microsoft range
1 Pfui 11:45 am Sep 23, 2019
 
0 12:00 am Jan 1, 1970
Mb2345Browser/9.0
I'm pretty sure this is actually a crawler, not a browser
8 notriddle 2:51 am Sep 10, 2019
Hakai
1 Pfui 3:57 am Sep 9, 2019
URI: ./ ./ mnt/ custom/ ProductDefinition
DVR remote code execution
1 Pfui 3:15 am Sep 8, 2019
serpstatbot/1.0 (advanced backlink tracking bot
3 notriddle 9:28 pm Sep 3, 2019
Moreover
4 lucy24 1:18 am Sep 2, 2019
BingPreview from Facebook
4 lucy24 8:50 pm Sep 1, 2019
GoogleImageProxy
redux; runs amok
9 Pfui 11:22 am Aug 30, 2019
looid.com Search Engine
3 Pfui 4:54 am Aug 29, 2019
RDDocuments
1 Pfui 2:25 pm Aug 25, 2019
Plague of Viewbots
help on how to detect view bots?
12 PeachyCheech 8:47 pm Aug 22, 2019
Looking at some strange hits from Microsoft IP's
Some are from outlook outbound mail servers?
2 SumGuy 6:27 am Aug 21, 2019
Firefox on Mac User-agent -> version number is 4-digit hex value
Seeing coordinated bot activity
4 SumGuy 1:38 am Aug 16, 2019
User Agent cdnposUserAgentcdnpos<>
3.224.211.xxx
2 Bewenched 8:22 pm Aug 13, 2019
Seeing hits from 3.0.0.0/8 for first time
Is Amazon
9 SumGuy 7:15 pm Aug 13, 2019
Bad Bot hack attempts 23.83.128.0/21
5 Bewenched 5:45 pm Aug 13, 2019
Instapaper/7.7.1.2 CFNetwork/978.0.7 Darwin/18.6.0
3 tangor 3:37 am Aug 3, 2019
newspaper
9 Pfui 10:32 pm Jul 29, 2019
Facebook pretending to be Bing
Bing UA from Facebook IP
8 dstiles 10:21 am Jul 29, 2019

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members