Welcome to WebmasterWorld Guest from 54.159.214.27

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Thousands of Spambot IPs Hitting my Site
Thousands of Spambot IPs Hitting my Site[3] ( 1 2 3 )
63 spiritualseo 2:54 am Feb 6, 2012
denied landing page
6 wilderness 9:41 pm Feb 5, 2012
Googlebot bloopers
"WebResult"
1 Pfui 1:04 am Feb 4, 2012
Novel User Agent
Made by ZmEu @ WhiteHat Team - www.whitehat.ro
3 dstiles 9:15 pm Feb 3, 2012
bigpond dot net dot au
bot or end-user?
8 cyberdyne 10:40 pm Feb 1, 2012
WebAir proxies
WebAir using compromised machines as proxies
5 dstiles 10:11 pm Feb 1, 2012
CMS Crawler
20 keyplyr 10:37 pm Jan 31, 2012
LinkSaver and LinkParser
wikimedia bots?
9 dstiles 9:58 pm Jan 31, 2012
Pagepeeker.com
bad bot
9 cyberdyne 11:41 am Jan 31, 2012
ldspider
code.google.com
3 Pfui 11:53 pm Jan 27, 2012
WBSearchBot
8 Pfui 10:45 pm Jan 27, 2012
HTTwaddle
6 lucy24 11:22 am Jan 25, 2012
ucsb seclab crawler
8 Pfui 9:02 pm Jan 22, 2012
Vancouver 0.9
3 MxAngel 10:45 am Jan 20, 2012
AWS look out.
Not a spider report, a warning of increased activity!
1 tangor 6:26 am Jan 19, 2012
SpiderLing
2 Pfui 6:58 am Jan 18, 2012
Gigabot Revisited
4 Pfui 5:53 am Jan 18, 2012
utopic
1 Pfui 7:07 pm Jan 17, 2012
obot
IBM iss.net IPs
5 Pfui 4:55 am Jan 16, 2012
EuripBot
3 keyplyr 8:46 pm Jan 14, 2012
MS Crawler Hiding as a Browser
14 incrediBILL 4:19 am Jan 11, 2012
Is this a valid user agent? and how do I block it?
Is this a valid user agent? and how do I block it?
12 spiritualseo 4:59 am Jan 10, 2012
Nook
2 Pfui 7:22 pm Jan 8, 2012
Servers using open proxies
4 dstiles 3:21 pm Jan 7, 2012
boia.org
tools request, ignore robots.txt
2 Pfui 7:06 am Jan 7, 2012

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month