Welcome to WebmasterWorld Guest from 54.198.147.79

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
LightningDownload/1.0.1
Should this be labeled a good bot or a bad bot?
2 Pushycat 3:37 pm Aug 15, 2002
Mirago robot
has it changed name?
9 allybongo 12:31 pm Aug 14, 2002
Ideas on Banning This
Scanning Attemp
11 frontpage 7:49 am Aug 14, 2002
Hits From Novell Border Manager
Why exactly would it hit?
4 Grumpus 10:45 pm Aug 13, 2002
Unknown spider with unsavoury name
f*****bot larbin2.5.0@unspecified.mail
3 shelleycat 11:02 am Aug 13, 2002
InfoSeek Japan's UAs and IPs?
6 littleman 3:56 am Aug 12, 2002
SetEnvIf User-Agent
How to Ban Bots with this Tool
30 frontpage 1:14 pm Aug 11, 2002
Open Invitation
making webbots/spiders id-ing easier
7 transistor 2:34 pm Aug 10, 2002
Orangebot
Who's this cheeky little fella then?
12 Nick_W 2:17 pm Aug 10, 2002
211.154.211.209, again.
8 Key_Master 4:00 am Aug 10, 2002
DigExt
Is this a spider?
5 allybongo 6:11 pm Aug 9, 2002
Vobsub
anyone else seen this?
3 Crazy_Fool 4:09 pm Aug 9, 2002
OrangeBot sniffing about yesterday
5 engine 11:12 am Aug 9, 2002
UdiSearch
coming from Yahoo
2 Key_Master 6:17 am Aug 9, 2002
211.154.211.209
heads up, spider faking user agent Googlebot
5 Key_Master 1:42 am Aug 9, 2002
ah-ha.com crawler
Why are they crawling, i donīt use them and they are pure PPC engine?
5 lazerzubb 8:06 pm Aug 8, 2002
Scooter-3.2.TF
Is this from Altavista
2 allybongo 5:31 pm Aug 8, 2002
trek29.sv.av.com As an IP Address Making Requests
Is this AV, or someone else?
4 Eric_Lander 12:00 am Aug 8, 2002
Is this a spider?
208.184.139.82.speedera.com
2 egghere 6:22 pm Aug 6, 2002
mogimogi/1.0
Anyone know this one?
4 korkus2000 6:18 pm Aug 5, 2002
Flickbot and Symantec Notify Internet
Anyone have any info
2 madamjujujive 3:26 am Aug 3, 2002
Whois Cogent Communication Inc.?
I get spidered by these guys a lot - who are they really?
12 jrota 2:21 pm Aug 2, 2002
larbin
bringing up an old topic
7 Big_Balou 7:22 am Aug 2, 2002
Zombie crawlers with a referrer sig?
We're getting hit from various places
11 Everyman 8:05 am Jul 31, 2002
scanner.infoseek.com
204.202.132.19 is touching me all over...
3 EliteWeb 11:00 pm Jul 30, 2002