Welcome to WebmasterWorld Guest from 54.145.44.134

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Possible bot
4 Readie 9:50 pm Jun 26, 2013
MSNBot posing as Internet Explorer 7 and not reading robots.txt?
11 robzilla 2:01 am Jun 26, 2013
June 2013 - Blocking Bad Spiders / Bots / User Agents
26 Imaster 2:15 am Jun 25, 2013
ixquick, duckduckgo, startpage
IP ranges of these search engines
5 motorhaven 10:40 pm Jun 24, 2013
Bad behavior from Microsoft IP
11 keyplyr 4:06 pm Jun 24, 2013
Bots coming from Peg Tech
198.2.192.0/18
6 lucy24 8:53 pm Jun 22, 2013
good riddance to bad robots
3 lucy24 6:25 pm Jun 17, 2013
Request Protocol
12 wilderness 9:28 pm Jun 13, 2013
Front Range & Pagesinvenotry
3 Ken_S 9:08 pm Jun 7, 2013
Goodby old "friends" - Hello Watson
I.B.M. buys SoftLayer for 2bln
2 blend27 11:50 am Jun 6, 2013
Fortinet Technologies
3 wilderness 11:51 pm Jun 3, 2013
Swapdrive Norton
2 wilderness 8:02 pm Jun 3, 2013
Strange Googlebot behavior
Grabbing same page over and over in one hit
7 grandma_genie 4:57 pm May 31, 2013
oBot robot where art thou?
5 lucy24 9:16 pm May 30, 2013
IANA Country Codes
No longer published as country codes?
10 dstiles 9:22 pm May 27, 2013
Needle
Ouch, here is a thread for you.
11 blend27 9:19 pm May 27, 2013
Rackspace
198.61.128.0 - 198.61.255.255
7 Bewenched 9:18 pm May 27, 2013
true.net business internet or hosting?
7 incrediBILL 7:18 pm May 17, 2013
above the law? ignores ROBOTS.TXT
13 lucy24 10:41 am May 17, 2013
Amazon AWS Hosts Bad Bots
amazonaws.com[3] ( 1 2 3 )
88 Pfui 11:07 pm May 16, 2013
AcoonBot (Leaseweb Germany)
37.58.52.15
7 Bewenched 9:44 pm May 12, 2013
Cyberbunker
9 keyplyr 7:00 am May 11, 2013
Firefox Browser just passing through
3 lucy24 6:50 pm May 8, 2013
Wotbox/2.0 is cheating from DSL hosting
Contains unauthorized content
2 incrediBILL 8:18 pm May 6, 2013
Server Farms - March 2013
Ongoing WMW server farm report[4] ( 1 2 3 4 )
92 wilderness 8:15 pm May 5, 2013

Featured Threads

Hot Threads This Week

Hot Threads This Month