Welcome to WebmasterWorld Guest from 54.145.122.109

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Performance Systems International
Who knows what?
2 carfac 8:17 pm Apr 19, 2003
Are these the correct robot names?
1 Jesse_Smith 6:35 pm Apr 19, 2003
anyway i can ban all bots?
4 indiandomain 11:46 am Apr 19, 2003
Robot with UA 'Robot'
1 drbrain 9:54 pm Apr 18, 2003
Blocking Monitoring Services
looking for list of ips.[2] ( 1 2 )
38 Brett_Tabke 4:54 pm Apr 18, 2003
Google Viewer
UA Id
1 wilderness 4:08 am Apr 18, 2003
List of search engine robots?
2 Jesse_Smith 10:36 pm Apr 17, 2003

20 nativenewyorker 10:55 am Apr 17, 2003
NetNose
A new engine?
6 volatilegx 9:57 pm Apr 16, 2003
IUSA Browser?
Doesn't get robots.txt
5 kewlbeezer 7:04 pm Apr 16, 2003
Honeypot concept for IIS, already done?
Has anyone written yet, global.asa to block ignoring robots.txt spiders?
5 shadowspawn 7:04 pm Apr 16, 2003
Guess What I want?
65.240.143.205
7 wilderness 7:55 am Apr 16, 2003
Mediapartners-Google/2.1
anyone seen this one? getting hammered
16 acronym 8:23 am Apr 15, 2003
new spiders to me anyone know why
two new spiders but why theyare crawling
2 chompy 7:44 am Apr 15, 2003
is 64.171.96.89 a bot?
ip spam on server
4 indiandomain 6:24 am Apr 15, 2003
HTTPLib/1.0
whats this guy?
2 Andrue 5:28 am Apr 15, 2003
no UA but then a: Mozilla/5.0 (Macintosh UA
Mozilla/5.0 (Macintosh
4 Andrue 4:53 am Apr 15, 2003
PF Free Web Search Tool
Another one to ban...<sigh>
5 pendanticist 4:14 am Apr 15, 2003
Way to "block" userAgents with ASP.NET
Just a little script I have written..
6 kewlbeezer 6:37 pm Apr 14, 2003
spiders agent names
11 bignooz 10:37 am Apr 14, 2003
Do you guys ban Cyveillance?
They don't behave very well sometimes
9 kewlbeezer 10:21 am Apr 14, 2003
Thai spider, or something else?
wierd hits from asianet.co.th
7 Ankheg 2:47 am Apr 14, 2003
Mac Finder 1.0.39
How about this crawler?
7 kewlbeezer 11:58 pm Apr 13, 2003
Googlebots being spoofed?
How can you tell?
14 pendanticist 9:41 pm Apr 10, 2003
List of 'BAD' bots?
Which one are they?
3 le_gber 8:57 pm Apr 10, 2003