Welcome to WebmasterWorld Guest from 54.167.177.207

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
powering the semantic web? invention-machine.com
they've been spidering with IP 12.30.60.78
1 han solo 4:44 pm Dec 15, 2000
Does sympatico.ca run spiders?
64.228.216.145 Mozilla/4.5 [en] (Win98; I)
6 han solo 5:19 pm Dec 13, 2000
Who's This?
atrax2.pa-x.dec.com (Mercator-2.0)
3 mivox 1:24 pm Dec 12, 2000
Ink Great White Crawl
2 Brett_Tabke 9:28 pm Dec 8, 2000
New Inktomi Japan User Agent
3 littleman 2:47 pm Dec 8, 2000
ip 64.5.69.112 user agent Java1.2.2 ideas?
spider's been after me, but why?
4 han solo 2:44 pm Dec 8, 2000
Anyone seen this & have any info?
9 engine 1:16 pm Dec 8, 2000
63.210.161.34 Slightly less annoying...
Microsoft URL Control: 33 pages in 20 secs?
3 mivox 6:48 pm Dec 7, 2000
Obnoxious behavior from 63.86.155.41
Can't track it down...
6 mivox 6:46 pm Dec 7, 2000
Fast? Crawler
Strange
5 BoneHeadicus 3:14 pm Dec 3, 2000
Microsoft URL Control
what was it that littleman said?
2 han solo 5:52 pm Nov 30, 2000
possible spider...
216.161.154.93 ua Mozilla/4.0+(compatible;+MSIE+5.0;+Windows
7 han solo 2:50 pm Nov 30, 2000
Ink / PositionTech spider
1 oLeon 2:04 pm Nov 30, 2000
Is this ink?
62.252.160.5 ip looks up to inktomi2-lan.server.ntl.com
7 han solo 4:38 pm Nov 29, 2000
Smart Whois
2 BoneHeadicus 4:27 pm Nov 28, 2000
Apache Guardian
2 BoneHeadicus 8:26 pm Nov 25, 2000
LexiBot/1.00
who is this?
3 han solo 2:29 pm Nov 24, 2000
JennyBot
Who is she?
8 scott 11:13 am Nov 24, 2000
PJSPider
2 Abe 4:13 pm Nov 21, 2000
SpaceBison/0.01
any ideas/thoughts?
4 han solo 3:15 pm Nov 21, 2000
Two spider owners that need slapped
2 Brett_Tabke 11:53 am Nov 21, 2000
12.25.194.37 is a surfer, or spider?
tracert shows velocity isp company...
4 han solo 4:58 pm Nov 20, 2000
Good identification spider book?
Spiders
7 Melanie 7:59 am Nov 19, 2000
Who is this
cm-24-161-36-192.nycap.rr.com
2 stcrim 8:47 am Nov 17, 2000
mass banning spiders
I am considering blocking all spiders...
3 han solo 2:28 pm Nov 16, 2000

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month