homepage Welcome to WebmasterWorld Guest from 54.211.100.183
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification

Search Engine Spider and User Agent Identification

Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL Forum Options: Reset Last Read Pointers

PreModerated forum to id search engine spiders

 
new Subject Messages Started by Last Message
newpost thread iconUsing mod_rewrite to stop bad spiders
  Where is it?
5transistor 12:47 am on May 30, 2002 (gmt 0)
newpost thread iconWhat is too much...
 
2opulyst 11:55 pm on May 29, 2002 (gmt 0)
newpost thread iconLegit Spider Usage?
  Are there ways I can use spiders that don't cause problems for site owners?
2OpeRabbit 8:40 pm on May 29, 2002 (gmt 0)
newpost thread iconIdentifying These IP's
  Totally Skewed my Log Report
16JanCounselor 6:23 pm on May 29, 2002 (gmt 0)
newpost thread iconAtspider
  Spam spider gone mad
2estaquieto 2:57 pm on May 28, 2002 (gmt 0)
newpost thread iconNew Spider?
  Gigabot/1.0
2dcheney 9:15 am on May 25, 2002 (gmt 0)
newpost thread iconWin32; WinHttp.WinHttpRequest.5
  67.113.225.66
4wilderness 12:39 am on May 25, 2002 (gmt 0)
newpost thread iconBlock and Complain success
  Sometimes they do clean up their act
5jdMorgan 5:31 am on May 24, 2002 (gmt 0)
newpost thread iconInteresting spiders from AV & Mercator
 
17Air 3:47 pm on May 23, 2002 (gmt 0)
newpost thread iconCruizer IV
  Does anyone know it
3olwen 8:38 am on May 22, 2002 (gmt 0)
newpost thread iconWhat is this user agent?
  lwp-request/2.01
3korkus2000 9:15 pm on May 21, 2002 (gmt 0)
newpost thread iconZealbot
  LookSmart Spider
2Key_Master 6:43 am on May 21, 2002 (gmt 0)
newpost thread iconHow do I disallow spiders?
  I want to know what code to put on a page to not allow spiders to crawl.
2Mitch 5:03 pm on May 20, 2002 (gmt 0)
newpost thread iconanother one to deny
 
7littleman 5:31 am on May 20, 2002 (gmt 0)
newpost thread iconTcl http client package 1.0
  What is this?
4misosoph 11:22 am on May 19, 2002 (gmt 0)
newpost thread iconAnyone Seen This INK bot
 
3dstanovic 11:01 am on May 19, 2002 (gmt 0)
newpost thread iconAsk Jeeves spider?
 
2korkus2000 1:30 pm on May 18, 2002 (gmt 0)
newpost thread iconAnyone seen this Ask Jeeves spider?
  Ask Jeeves spider - why?
6IanTurner 1:55 am on May 18, 2002 (gmt 0)
newpost thread iconCoding a spider
  How to give yourself an agent name
6chris_f 6:53 pm on May 17, 2002 (gmt 0)
newpost thread iconInktomi
  New C class or did I forget to add this one again?
2msgraph 4:47 pm on May 17, 2002 (gmt 0)
newpost thread iconwebcollage variation
  collage.cgi/1.82
4misosoph 2:09 pm on May 17, 2002 (gmt 0)
newpost thread iconUnknown Crawler
  193.95.83.130
9chris_f 8:07 am on May 15, 2002 (gmt 0)
newpost thread iconspider-loh-tc063.proxy.aol.com
  What Is This
5Funfone 8:44 pm on May 14, 2002 (gmt 0)
newpost thread iconAltaVista Spider?
  trek17.sv.av.com
3HyperGeek 4:27 pm on May 14, 2002 (gmt 0)
newpost thread iconUnknown - Lots of activity
  66.28.250.176
3HyperGeek 4:02 pm on May 14, 2002 (gmt 0)
Displaying Page 184 of 216<< Back 1 ... 174 175 176 177 178 179 180 181 182 183 [ 184 ] 185 186 187 188 189 190 191 192 193 ... 216 Next >>
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved