Welcome to WebmasterWorld Guest from 54.144.48.252

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Fast WebCrawler 3.6
Is this new?
4 coconutz 6:27 pm Jun 9, 2002
66.150.40.220
Can anyone identify it?
5 Birdman 11:25 am Jun 9, 2002
FirstGov.gov Search - POC:firstgov.webmasters@gsa.gov
Is this for real?
5 Key_Master 5:15 am Jun 9, 2002
Brainysearch
64.57.64.90
1 volatilegx 11:45 pm Jun 7, 2002
Snoopy_v0.1
4 korkus2000 3:34 pm Jun 5, 2002
BlitzBOT@tricus.net ?
3 bull 3:54 pm Jun 4, 2002
65.24.37.57 - toto
toto? - no referrer, no ua
6 misosoph 4:32 am Jun 4, 2002
Unwanted Spiders - how to limit/kill access
simple ways to code, spiders list etc.
3 lak12 7:56 pm Jun 3, 2002
scooter/1.0 (netscan.fr)
Is this Alta Vista?
3 volatilegx 5:51 pm Jun 3, 2002
Monitoring bot activity with a script?
8 Bluestreak 1:00 am Jun 3, 2002
Charter
My apologies
1 wilderness 9:54 pm Jun 2, 2002
autoemailspider
email harvester or poorly named spider?
3 zem 12:48 am Jun 2, 2002
DiaGem
Another bot to ban
6 Son_House 6:03 am Jun 1, 2002
216.200.130.207
Ask Jeeves Spider?
3 Key_Master 7:15 pm May 31, 2002
new google spider IP?
64.95.79.195
5 chiefmonkey 5:44 pm May 31, 2002
NetResearchServer/2.2
Another new name for IncyWincy
2 mbauser2 7:04 am May 31, 2002
Alexa impersonator?
Is someone faking an Alexa spider?
6 Ralph_Slate 10:20 am May 30, 2002
new FAST IPs
just found these
7 volatilegx 9:10 am May 30, 2002
Is this a spider
messes up my logfiles
4 MarkArt 4:01 am May 30, 2002
Using mod_rewrite to stop bad spiders
Where is it?
5 transistor 12:47 am May 30, 2002
What is too much...
2 opulyst 11:55 pm May 29, 2002
Legit Spider Usage?
Are there ways I can use spiders that don't cause problems for site owners?
2 OpeRabbit 8:40 pm May 29, 2002
Identifying These IP's
Totally Skewed my Log Report
16 JanCounselor 6:23 pm May 29, 2002
Atspider
Spam spider gone mad
2 estaquieto 2:57 pm May 28, 2002
New Spider?
Gigabot/1.0
2 dcheney 9:15 am May 25, 2002

Featured Threads

Hot Threads This Week

Hot Threads This Month