Welcome to WebmasterWorld Guest from 54.162.155.183

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Scrapers Using Peak Web Hosting and Generic Browser User Agents
4 keyplyr 10:53 pm May 19, 2012
deny all
15 wilderness 3:50 pm May 14, 2012
Yahoo SE
2 wilderness 1:46 pm May 14, 2012
Bing blows a gasket
9 wilderness 7:57 pm May 13, 2012
PycURL
4 wilderness 7:00 pm May 13, 2012
Cookie based spider catcher
6 motorhaven 11:16 pm May 12, 2012
Been getting 200+ page loads per day from same IP
recorded by js based analytics
10 scooterdude 7:01 pm May 12, 2012
Digital AlphaServer User Agent
3 wilderness 6:26 pm May 12, 2012
Bright House Networks
3 wilderness 11:21 pm May 10, 2012
HopOne
10 keyplyr 9:17 pm May 10, 2012
MediaTemple
3 keyplyr 8:58 pm May 10, 2012
Multiple User Agents, same IP
Is this a bot?
17 grandma_genie 8:49 pm May 10, 2012
User Agent "-"
missing User Agent
8 dupres01 8:01 pm May 9, 2012
meet the pdfbot
New Variation of Googlebot?
3 lucy24 7:48 pm May 9, 2012
More Multi-National Attempts from Avant Browser and IE 5 on Win 3.1
11 incrediBILL 8:41 pm May 8, 2012
Moved: block spammers using proxies?
Post was moved here: http://www.webmasterworld.com/apache/4451068.htm
1 System 3:55 pm May 8, 2012
The Onion Strikes - TOR Proxy Comment Spammers
tor exit node drive-by
9 iamzippy 6:57 pm May 3, 2012
open proxies
7 wilderness 12:12 pm May 3, 2012
MojeekBot
14 wilderness 2:16 pm May 2, 2012
fatlens.com using Googlebot agents?
5 brokaddr 1:19 pm May 2, 2012
Besthosting
3 keyplyr 9:46 pm Apr 29, 2012
Eonix/Infinity
1 keyplyr 11:05 pm Apr 28, 2012
EGIHosting
4 keyplyr 2:20 pm Apr 28, 2012
WeblexBot
8 keyplyr 7:26 pm Apr 27, 2012
*missing* - user agent from Yahoo, or spoof?
7 brokaddr 3:13 am Apr 27, 2012

Featured Threads

Hot Threads This Week

Hot Threads This Month