Welcome to WebmasterWorld Guest from 54.160.163.163

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Inktomi
just added a new Class C
5 volatilegx 3:44 pm Apr 30, 2002
Need help with blocking this agent
4 willtell 5:41 pm Apr 29, 2002
Ng/1.0
belongs to Exalead
3 volatilegx 5:38 pm Apr 25, 2002
Microsoft
interesting
4 wilderness 5:14 am Apr 25, 2002
XO Communications?
Who are these folks?
12 Friday 4:50 pm Apr 24, 2002
AOL Spider
AC85944C.ipt.aol.com
5 Friday 8:02 am Apr 24, 2002
Is this a bot or a browser?
2 korkus2000 2:57 am Apr 23, 2002
RPT-HTTPClient/0.3-3
Nameprotect.com
7 wilderness 2:33 am Apr 23, 2002
Search Engine Domain Identification
Not Spider Listings but URL Referral Info
5 YadiYudi 2:11 am Apr 23, 2002
spidernet
help in defining this one
2 EliteWeb 6:29 pm Apr 22, 2002
Inktomi UA
was this out?
2 msgraph 1:00 am Apr 21, 2002
DOD Web Scans
Department of Defense Robots
4 hbarker 10:52 pm Apr 20, 2002
C-Block-192
Who is this?
9 sdtex 6:44 pm Apr 19, 2002
Who is this?
I couldn't find any info on it.
3 RussellC 3:15 pm Apr 19, 2002
What is This?
Micro$oft?
3 frontpage 12:14 pm Apr 19, 2002
New to SEO and robots
robots.txt for multiple sites on one server
2 dabrisz 10:11 pm Apr 17, 2002
spiders!
unknown spider species
7 mikeala_2000 6:07 pm Apr 17, 2002
unknown from exodus
216.33.229.162
2 berno 1:36 pm Apr 17, 2002
trafficfixer.com
advertisment in the logfiles
6 skirril 7:47 am Apr 17, 2002
Teoma's Spider
Does anyone have any idea on this one?
4 dvb_99 7:43 pm Apr 16, 2002
New Scooter UA
8 msgraph 10:06 am Apr 14, 2002
new ua: Viking/1.8
3 skirril 5:03 pm Apr 13, 2002
Blocking UA "-"
5 skirril 12:24 am Apr 13, 2002
htdig
6 wilderness 10:31 pm Apr 12, 2002
Heads Up
6 toolman 8:40 am Apr 12, 2002

Featured Threads

Hot Threads This Week

Hot Threads This Month