homepage Welcome to WebmasterWorld Guest from 54.211.97.242
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification

Search Engine Spider and User Agent Identification

Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL Forum Options: Reset Last Read Pointers

PreModerated forum to id search engine spiders

 
new Subject Messages Started by Last Message
newpost thread iconSpiders I Hate
  I really HATE to see this in my logs!
13scott 8:56 pm on Jan 14, 2001 (gmt 0)
newpost thread iconNewbie - Can spiders go bezerk?
  This spider sucks the same page over and over again
8Macguru 12:07 am on Jan 14, 2001 (gmt 0)
newpost thread iconSc2_Merc_resh_37_1_C-1.0
  Is this new
2BoneHeadicus 6:10 pm on Jan 13, 2001 (gmt 0)
newpost thread iconA new visitor from Google?
  Hopefully the unblocking agent!
5makemetop 9:23 am on Jan 13, 2001 (gmt 0)
newpost thread iconinktomi1-mid.server.ntl.com
  whodat?
3BoneHeadicus 12:05 am on Jan 12, 2001 (gmt 0)
newpost thread iconRobot from Cisco Systems?
 
6msgraph 6:07 pm on Jan 10, 2001 (gmt 0)
newpost thread iconWhatzup?
 
23BoneHeadicus 11:11 pm on Jan 8, 2001 (gmt 0)
newpost thread iconWhat spider is this?
  ru-robot
4merlin78 4:54 am on Jan 8, 2001 (gmt 0)
newpost thread iconNew sighting: JennyBOt/0.1
 
3skirril 2:48 am on Jan 8, 2001 (gmt 0)
newpost thread iconvspider
 
2Steph 2:08 am on Jan 8, 2001 (gmt 0)
newpost thread iconFast's Mozilla spider
 
11littleman 7:07 pm on Jan 6, 2001 (gmt 0)
newpost thread iconNewbie - what is cr001.digital-integrity.com?
  Are we beeing checked for cloaking?
5Macguru 8:24 am on Jan 6, 2001 (gmt 0)
newpost thread iconDirectHit
  new ips
5PeteU 9:11 pm on Jan 5, 2001 (gmt 0)
newpost thread iconSlurp or what ?
  suspicious spider
4ulstrup 3:05 pm on Jan 4, 2001 (gmt 0)
newpost thread iconGooglebot IP's
 
52_much 12:52 pm on Jan 4, 2001 (gmt 0)
newpost thread iconExcite is out of hibernation
 
1littleman 7:28 pm on Dec 27, 2000 (gmt 0)
newpost thread iconMARS SV user agent
 
3msgraph 10:45 am on Dec 22, 2000 (gmt 0)
newpost thread iconAverage traffic percentage of spider visits
 
15WebRookie 8:08 pm on Dec 19, 2000 (gmt 0)
newpost thread iconoBot ((compatible;Win32))
 
3Steph 3:13 pm on Dec 19, 2000 (gmt 0)
newpost thread iconWho is this?
  avfwclient.sv.av.com #UA roach.smo.av.com-1.0
1oLeon 10:54 am on Dec 19, 2000 (gmt 0)
newpost thread iconetchels-indexer?
  Anyone knows this spider?
2skirril 1:18 am on Dec 19, 2000 (gmt 0)
newpost thread iconDoes anyone know which spiders get confused by frames?
 
2White_Rabbit 10:26 pm on Dec 18, 2000 (gmt 0)
newpost thread iconoddball do-it-yourself SE spiders
  cfw3.rdc1.nsw.excitehome.net.au
1mivox 9:05 pm on Dec 18, 2000 (gmt 0)
newpost thread iconPaying for Inktomi? You need to see this...
  Faked user agent for Inktomi's pay for play
1djuice999 10:49 pm on Dec 15, 2000 (gmt 0)
newpost thread iconAnother Slurp Mozilla spider
 
5msgraph 7:11 pm on Dec 15, 2000 (gmt 0)
Displaying Page 209 of 216<< Back 1 ... 199 200 201 202 203 204 205 206 207 208 [ 209 ] 210 211 212 213 214 215 216 Next >>
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved