Welcome to WebmasterWorld Guest from 54.196.231.129

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Server Farms - January 2016
Continuation of the discussion of hosting company IP ranges
24 incrediBILL 10:14 pm Feb 11, 2016
Default User Agents of Programming Libraries and Command Line Tools
Resource page for common user agents
15 incrediBILL 8:54 pm Apr 29, 2008
 
0 12:00 am Jan 1, 1970
The Donald web spider user agent
it had to happen.
10 lucy24 12:34 pm Feb 11, 2016
Searchie
4 keyplyr 6:28 pm Feb 10, 2016
PrivacyAwareBot
3 w3bmastine 8:03 am Feb 10, 2016
Weberknecht
2 lucy24 10:47 am Feb 8, 2016
apple.bot
massive requests
15 johnhh 11:07 pm Feb 7, 2016
Oddly suspicious traffic claiming duckduckgo
12 grouchy_sysadmin 4:39 am Feb 7, 2016
BellPagesCA
4 w3bmastine 4:15 am Feb 7, 2016
Putin
4 keyplyr 3:20 am Feb 6, 2016
mfibot/1.1
2 w3bmastine 10:46 am Feb 4, 2016
YYSpider
3 keyplyr 10:23 am Feb 4, 2016
Experibot v1
[2] ( 1 2 )
34 keyplyr 12:33 pm Feb 2, 2016
Where am I?
12 lucy24 3:30 am Feb 2, 2016
Dazoobot/
1 keyplyr 10:12 pm Feb 1, 2016
Donuts Content Explorer
My website was visited by the Donuts Content Explorer.
1 w3bmastine 1:42 pm Feb 1, 2016
eContext
1 keyplyr 8:09 am Feb 1, 2016
SafeSearch microdata crawler
safesearch.avira.com seems to have a SafeSearch microdata crawler.
1 w3bmastine 5:11 pm Jan 30, 2016
Chilkat
1 keyplyr 10:34 am Jan 28, 2016
wildfetcher
1 analognico 10:06 am Jan 28, 2016
Convincing IP spoofing / user simulation
Pointers to detection of IP spoofing[3] ( 1 2 3 )
61 Simon_H 7:29 am Jan 28, 2016
etoolsbot
4 keyplyr 10:11 pm Jan 26, 2016
ICC-Crawler
1 keyplyr 6:46 pm Jan 26, 2016
Kraken/0.1
1 grouchy_sysadmin 3:54 pm Jan 26, 2016