Welcome to WebmasterWorld Guest from 3.229.122.219

Forum Moderated by: Ocean10000

Search Engine Spider and User Agent Identification


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Server Farms - July 2018
Continuing discussion of hosting company IP ranges[5] ( 1 2 3 4 5 )
134 keyplyr 3:24 am Nov 21, 2019
Robot's User Agent String
What webmasters want
1 keyplyr 9:34 pm Oct 14, 2018
At Home with the Robots: 2017 edition
Critique of the Year's Active User Agents
15 lucy24 3:52 pm May 18, 2017
Blocking Methods
How to block access to your server
1 keyplyr 6:29 pm May 10, 2017
User Agents of Programming Libraries and Command Line Tools
Common user agents
14 incrediBILL 8:54 pm Apr 29, 2008
 
0 12:00 am Jan 1, 1970
woorank
1 lucy24 9:16 pm Dec 8, 2019
Corax
3 tangor 6:02 pm Dec 7, 2019
Bot Mashing
6 rivsrush 7:55 am Dec 7, 2019
Linespider
9 lucy24 11:16 am Nov 21, 2019
Any humans come from there?
*.compute.hwclouds-dns.com
4 blend27 6:02 am Nov 17, 2019
ApiTool
7 Pfui 2:26 am Nov 17, 2019
Python, Curl and Robots.txt
17 dstiles 8:48 pm Nov 12, 2019
Google Adsense Preview
1 JorgeV 9:04 am Oct 22, 2019
whisper
14 lucy24 10:00 am Oct 20, 2019
Return / continuation of msnbot
search.msn.com/msnbot.html
11 dstiles 9:58 am Oct 20, 2019
Google hiccup
3 wilderness 9:33 am Oct 14, 2019
Google-Read-Aloud
1 lucy24 8:48 pm Oct 6, 2019
Hi
Pages exploit UA
15 Pfui 5:26 am Oct 5, 2019
URI: ./ ./ mnt/ custom/ ProductDefinition
DVR remote code execution
3 Pfui 5:35 pm Sep 27, 2019
Yisou again
2 lucy24 12:06 am Sep 27, 2019
GrumpyCrawler
Microsoft range
3 Pfui 6:02 pm Sep 24, 2019
Thinking of blocking 40.64.0.0/10 and 51.140.0.0/14 (MSFT)
Seen some bot stuff recently
2 SumGuy 5:53 am Sep 24, 2019
myAgent
Microsoft range
1 Pfui 11:45 am Sep 23, 2019
Mb2345Browser/9.0
I'm pretty sure this is actually a crawler, not a browser
8 notriddle 2:51 am Sep 10, 2019

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members