Welcome to WebmasterWorld Guest from 54.90.185.120

Forum Moderated by: Ocean10000 & keyplyr

Search Engine Spider and User Agent Identification


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Server Farms - July 2018
Continuing discussion of hosting company IP ranges[4] ( 1 2 3 4 )
98 keyplyr 3:33 pm Nov 14, 2018
Robot's User Agent String
What webmasters want
1 keyplyr 9:34 pm Oct 14, 2018
At Home with the Robots: 2017 edition
Critique of the Year's Active User Agents
15 lucy24 3:52 pm May 18, 2017
Blocking Methods
How to block access to your server
1 keyplyr 6:29 pm May 10, 2017
User Agents of Programming Libraries and Command Line Tools
Common user agents
14 incrediBILL 8:54 pm Apr 29, 2008
 
0 12:00 am Jan 1, 1970
GoogleDocs
1 ClosedForLunch 8:29 pm Nov 13, 2018
iframe-toloka.com
Yandex
1 ClosedForLunch 2:32 pm Nov 6, 2018
facebookexternalhit
3 lucy24 10:34 pm Nov 4, 2018
Falkon
5 lucy24 9:38 am Oct 31, 2018
OutclicksBot
new UA, new range
12 keyplyr 9:04 pm Oct 30, 2018
Apache-HttpAsyncClient
1 keyplyr 8:59 pm Oct 30, 2018
adplexity
1 keyplyr 7:36 pm Oct 29, 2018
weborama-fetcher
1 keyplyr 2:38 am Oct 25, 2018
PocketImageCache
2 keyplyr 3:11 pm Oct 24, 2018
Aranea Web Corpora Project
3 keyplyr 6:41 am Oct 24, 2018
GSC >Crawl Errors >Server Connectivity
5 keyplyr 3:23 am Oct 24, 2018
Android's Mobile Chrome "Show simplified view"
undetectable in UA string
8 keyplyr 1:17 am Oct 23, 2018
Keepsafe Browser
1 keyplyr 10:57 pm Oct 22, 2018
BublupBot
1 keyplyr 2:44 am Oct 22, 2018
HttpUrlConnection
Jersey
1 keyplyr 8:31 pm Oct 21, 2018
Question about Bing Preview
Bing preview being used as a proxy?
2 SumGuy 4:59 pm Oct 21, 2018
Default User Agent
5 keyplyr 5:59 am Oct 19, 2018
WordPress.com Reader
2 keyplyr 9:35 pm Oct 18, 2018
Google adbot is now hitting my site
I don't buy or sell or connect to ad networks?
12 SumGuy 4:35 am Oct 18, 2018

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members