Welcome to WebmasterWorld Guest from 54.205.96.97

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
User-Agent: MyLycos
Please, this is all over my stuff
16 han solo 7:25 pm Feb 28, 2001
Ip:199.245.188.74
UA: libwww-perl/5.50
4 roscoepico 6:43 pm Feb 28, 2001
who is this?
5 hohmaster 5:30 pm Feb 28, 2001
Newbie - Do Spiders eat JPG files?
I could not identify this IP
8 Macguru 12:34 pm Feb 28, 2001
Who/What is this?
2 jk321 8:09 am Feb 27, 2001
Crawler6.bos2.fast-search.net
FAST / Alltheweb crawler?
5 Rumbas 5:28 am Feb 25, 2001
Make them stop!
any luck?
8 luckynh 1:46 pm Feb 21, 2001
211.117.61.68
4 Froggyman 8:45 pm Feb 20, 2001
Sidewinder
What is it doing?
3 Hope 6:50 pm Feb 20, 2001
JennyBot/0.1?
64.97.116.103; JennyBot/0.1
5 skirril 2:27 am Feb 20, 2001
Davesengine.com
1 msgraph 1:53 pm Feb 19, 2001
Netcraft..
ariston.netcraft.com ; Mozilla 4.0 Compat..
3 skirril 10:23 pm Feb 17, 2001
Where can I optain Spider List?
Where can I optain Spider List?
9 clienttell 12:51 pm Feb 16, 2001
Why are non search engines robots hitting our site?
2 Acternaweb 4:27 pm Feb 15, 2001
Clarifications on Marvin/1.0
was just a test?
1 skirril 9:35 am Feb 14, 2001
Spoofed Mozilla/4.72
Acts like a robot, doesn't send a Host header
2 Kimihia 3:09 am Feb 14, 2001
Robots Searching For .... ?
interesting question
4 zon2k 9:34 am Feb 13, 2001
Wonder what AV are testing
(is MC the new, hip AV)
5 vis 9:22 am Feb 13, 2001
Zyborg?
3 jlara 12:06 am Feb 13, 2001
who is InterNAP...
and why do they keep hitting /
6 mivox 7:36 pm Feb 12, 2001
German spiders
up to date list of german spiders
5 jan_zandbergen 11:18 am Feb 10, 2001
New Ink Spiders
9 BoneHeadicus 12:36 am Feb 9, 2001
Spider foot print
What does this term mean
9 startup 10:16 am Feb 5, 2001
Fast Spider?
Looks like Fast...smells like Fast...
7 BoneHeadicus 5:53 am Feb 4, 2001
who is this?
2 jord 5:11 am Feb 4, 2001

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month