Welcome to WebmasterWorld Guest from 23.20.147.6

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
List of 'BAD' bots?
Which one are they?
3 le_gber 8:57 pm Apr 10, 2003
Mozilla/3.0 (compatible) Again
Pulling strange urls
5 jimbeetle 8:17 pm Apr 10, 2003
65.92.240.232 - Is this a Googlebot?
Googlebot/1.0
8 Jesse_Smith 4:04 pm Apr 10, 2003
URL_Spider_Pro/3.0
Update to URL Spider Pro/1.92
2 pendanticist 5:42 am Apr 10, 2003
proxy.google.com
2 coffeelover 12:49 am Apr 10, 2003
timboBot/0.9
weblog indexer
2 frontpage 10:33 pm Apr 9, 2003
Industry Program 1.0.5
yet another guestbook/email harvester...
1 WitchLars 10:14 pm Apr 9, 2003
'index.html' and 'index.htm'
Will spiders see both if they are both in root directory?
2 TWhalen 9:51 pm Apr 9, 2003
bad bot or bad self made spider trap?
6 matthias 6:30 pm Apr 9, 2003
snorkel-x.markwatch.com
Is this a spider?
5 scone 6:04 pm Apr 9, 2003
AlkalineBOT
behaving a bit aggressive...
7 RonPK 9:17 am Apr 9, 2003
what's this (PHP/4.2.3)
5 fandangood 7:58 am Apr 9, 2003
What to do with names and I.P. addresses of spiders?
why spend a ton of time gathering info?
11 Brad1ey 1:06 am Apr 9, 2003
Windows NT 5.2?
I wasn't aware of this type of browser...
15 kewlbeezer 5:11 pm Apr 8, 2003
Weak bot
80.8.120.54
3 wilderness 12:45 am Apr 8, 2003
Larbin-experimental
Rude bot
5 herb 6:43 pm Apr 7, 2003
Google Visit
11 DavidT 5:31 pm Apr 7, 2003
net-sweeper
Ignored Robots.txt
10 frontpage 5:15 pm Apr 7, 2003
Microsoft Spider poking around?
seems to be behaving itself
7 WitchLars 9:11 am Apr 7, 2003
fresh vs deep ips
22 Brett_Tabke 4:43 am Apr 7, 2003
WebStripper
I know, you probably already have it banned...
4 carfac 7:11 pm Apr 6, 2003
WebFilter Robot 1.0
Good/Bad or just plain Ugly?
8 pendanticist 4:59 pm Apr 5, 2003
Four visited in one day
Is it unusual?
3 jaybee 4:33 pm Apr 5, 2003
NCSA_Beta_1
The Supercomputer is after me!
3 bcolflesh 4:48 am Apr 5, 2003
Cloaked Spider!?
...something strange went through my site!
3 frogg 12:41 am Apr 4, 2003

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members