Welcome to WebmasterWorld Guest from 107.20.54.98

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Does IP Range Block Via WHM Blacklist Block Same IP Ranges at cPanel
Does IP Block at WHM BruteForce Level Filter Down to Unique Domains on VPS?
12 Webwork 10:40 pm Jan 14, 2014
BRIC nations
Eating Bricks
4 Angonasec 7:27 am Jan 13, 2014
sputnik
6 lucy24 7:52 pm Jan 12, 2014
Zeerch
9 Angonasec 6:30 pm Jan 11, 2014
Rampant of Redmond
Shivering my Timbers
7 Angonasec 12:52 pm Jan 10, 2014
112.0.0.0/8
7 keyplyr 9:43 pm Jan 7, 2014
UNTRUSTED in Nokia User Agent
[2] ( 1 2 )
45 lucy24 11:41 am Jan 4, 2014
thunderstone
yes, them again
3 lucy24 6:28 pm Jan 1, 2014
Come on Yahoo. Really?
7 bobothecat2 8:38 pm Dec 27, 2013
who's on 168?
1 lucy24 3:10 am Dec 22, 2013
archive.org bot
bad bot
7 keyplyr 7:20 am Dec 20, 2013
Google Test-Bot: Google-Test2
11 doc_z 8:28 pm Dec 19, 2013
Google? Is that you?
13 lucy24 10:33 pm Dec 18, 2013
Is there a Browser. . .
10 wilderness 12:53 am Dec 16, 2013
TECH UPDATE: Javascript Makes Dumb Bots Obsolete
Javascript Must Be Crawled For Scrapers To Survive
3 incrediBILL 10:12 pm Dec 15, 2013
the plainclothes bingbot
8 lucy24 8:03 pm Dec 14, 2013
Yahoo:LinkExpander:Slingstone
7 keyplyr 2:06 pm Dec 13, 2013
Google Translate
7 wilderness 2:43 am Dec 11, 2013
YaBrowser
15 lucy24 10:04 am Dec 10, 2013
wscheck
1 lucy24 1:17 am Dec 10, 2013
this time it's Sogou
61.135 returns
1 lucy24 11:32 pm Dec 9, 2013
New stupid referer spammer
Referer spam via agent string (mistyped)
9 yaimapitu 2:58 pm Dec 8, 2013
rarely used
12 lucy24 9:22 am Dec 1, 2013
New Scraping Tool
9 keyplyr 9:07 am Nov 28, 2013
Bixocrawler
Another data miner
1 dstiles 4:02 pm Nov 24, 2013