Welcome to WebmasterWorld Guest from 54.145.166.96

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Static IP Services
4 wilderness 1:43 pm Mar 18, 2013
Server Farms, Elron Technologies and more
[6] ( 1 2 3 4 5 6 )
169 wilderness 12:24 am Mar 8, 2013
Temporary Absence Update
8 incrediBILL 1:57 am Mar 6, 2013
Revised Spider IPs and Other Details
At Home with the Robots: 2013 edition
6 lucy24 5:08 pm Mar 3, 2013
Facebook
2 topr8 8:20 am Feb 27, 2013
Softlayer
13 Ken_S 9:43 pm Feb 21, 2013
Pinterest Domain Verifier
6 keyplyr 11:50 pm Feb 20, 2013
another day, another translation
1 lucy24 6:46 am Feb 19, 2013
Blekko how to ban
5 viggen 12:41 am Feb 17, 2013
msnbot-media
What's a nice robot like you doing in a place like this?[2] ( 1 2 )
58 lucy24 4:25 am Feb 12, 2013
-very- much the same robot
10 lucy24 10:48 pm Feb 4, 2013
U of Cal
Santa Barbara
9 wilderness 11:19 am Feb 4, 2013
BingBot Fake
19 Ken_S 3:45 am Feb 4, 2013
Yandex gets fatter
13 lucy24 9:57 pm Feb 1, 2013
the latest impenetrable disguise
[2] ( 1 2 )
40 lucy24 3:47 am Jan 30, 2013
goo goo GET GET
8 lucy24 7:50 am Jan 27, 2013
Filtering Out Really Hard To Find Bad Bots
26 incrediBILL 2:16 am Jan 24, 2013
ZTE-Z431/1.4.0 NetFront/4.2 QTV5.1 User-Agent
Profile/MIDP-2.1 Configuration/CLDC-1.1
4 not2easy 9:24 pm Jan 23, 2013
strange g-refers
1 wilderness 5:41 pm Jan 22, 2013
WBSearchBot misbehaving
12 incrediBILL 12:11 am Jan 19, 2013
LLF user agent
1 Ken_S 6:32 pm Jan 18, 2013
MSOffice 12
What were they thinking?
1 blend27 5:54 pm Jan 18, 2013
and another gets on.
5 lucy24 1:25 pm Jan 18, 2013
Lots of interesting user agents
4 Bewenched 7:54 pm Jan 15, 2013
Baidu's IPs - Which Are Legit?
12 incrediBILL 7:21 pm Jan 13, 2013

Featured Threads

Hot Threads This Week

Hot Threads This Month