Welcome to WebmasterWorld Guest from 184.73.112.180

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
bingbot/3.0-alpha
11 Angonasec 8:18 am Mar 28, 2015
TosCrawler
toshiba
15 keyplyr 1:58 am Mar 28, 2015
alooflybot
aloofly.com
4 Pfui 9:12 pm Mar 27, 2015
pt-PT
12 trintragula 8:33 pm Mar 24, 2015
Netscan
1 Angonasec 12:43 pm Mar 23, 2015
MSIE or Firefox mutant; not on Windows server
10 aristotle 8:02 pm Mar 22, 2015
asp translation
12 lucy24 6:36 am Mar 20, 2015
SheerBoredom Bot
3 Angonasec 3:47 am Mar 19, 2015
BUbiNG
law-abiding Italian robot
6 lucy24 10:13 pm Mar 18, 2015
Is this a bot?
13 aristotle 10:10 pm Mar 18, 2015
Dataline
dtln.ru cloud servers
13 keyplyr 4:00 pm Mar 17, 2015
Wire/0.22
Brazil
1 keyplyr 10:55 am Mar 15, 2015
strange googlebot behavior
referer: webring; user agent: googlebot
4 dupres01 9:38 pm Mar 13, 2015
Exabot-Thumbnails redux
3 lucy24 9:29 pm Mar 13, 2015
Blocking all traffic from amazonaws.com
I want to block all traffic
15 Kratos 1:14 pm Mar 11, 2015
BingAdsAdInsightBot
New msnbot Spawn
7 Pfui 3:18 am Mar 11, 2015
iParadigms
TurnItIn
8 keyplyr 4:09 pm Mar 9, 2015
kb58.cn
Is there a bot to this
9 Marshall 1:37 pm Mar 7, 2015
Newbie: Need help with server farms and IP ranges
25 MrSavage 2:11 am Mar 6, 2015
PageFetcher-Google-CoOp
14 aristotle 2:04 am Mar 6, 2015
Gulper Web Bot
stonybrook.edu
8 keyplyr 4:43 am Mar 3, 2015
This thing shows up every 30-40 minutes on two of my sites
16 aristotle 2:09 am Mar 3, 2015
Image Bot Names?
4 keyplyr 10:46 pm Feb 27, 2015
Apple search engine bot?
24 johnhh 8:49 pm Feb 25, 2015
IANA Private CIDRs
Respectable Spooks only?
4 Angonasec 7:14 pm Feb 25, 2015

Featured Threads

Hot Threads This Week

Hot Threads This Month