homepage Welcome to WebmasterWorld Guest from 54.145.148.66
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification

Search Engine Spider and User Agent Identification

Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL Forum Options: Reset Last Read Pointers

PreModerated forum to id search engine spiders

 
new Subject Messages Started by Last Message
newpost thread iconBlock and Complain success
  Sometimes they do clean up their act
5jdMorgan 5:31 am on May 24, 2002 (gmt 0)
newpost thread iconInteresting spiders from AV & Mercator
 
17Air 3:47 pm on May 23, 2002 (gmt 0)
newpost thread iconCruizer IV
  Does anyone know it
3olwen 8:38 am on May 22, 2002 (gmt 0)
newpost thread iconWhat is this user agent?
  lwp-request/2.01
3korkus2000 9:15 pm on May 21, 2002 (gmt 0)
newpost thread iconZealbot
  LookSmart Spider
2Key_Master 6:43 am on May 21, 2002 (gmt 0)
newpost thread iconHow do I disallow spiders?
  I want to know what code to put on a page to not allow spiders to crawl.
2Mitch 5:03 pm on May 20, 2002 (gmt 0)
newpost thread iconanother one to deny
 
7littleman 5:31 am on May 20, 2002 (gmt 0)
newpost thread iconTcl http client package 1.0
  What is this?
4misosoph 11:22 am on May 19, 2002 (gmt 0)
newpost thread iconAnyone Seen This INK bot
 
3dstanovic 11:01 am on May 19, 2002 (gmt 0)
newpost thread iconAsk Jeeves spider?
 
2korkus2000 1:30 pm on May 18, 2002 (gmt 0)
newpost thread iconAnyone seen this Ask Jeeves spider?
  Ask Jeeves spider - why?
6IanTurner 1:55 am on May 18, 2002 (gmt 0)
newpost thread iconCoding a spider
  How to give yourself an agent name
6chris_f 6:53 pm on May 17, 2002 (gmt 0)
newpost thread iconInktomi
  New C class or did I forget to add this one again?
2msgraph 4:47 pm on May 17, 2002 (gmt 0)
newpost thread iconwebcollage variation
  collage.cgi/1.82
4misosoph 2:09 pm on May 17, 2002 (gmt 0)
newpost thread iconUnknown Crawler
  193.95.83.130
9chris_f 8:07 am on May 15, 2002 (gmt 0)
newpost thread iconspider-loh-tc063.proxy.aol.com
  What Is This
5Funfone 8:44 pm on May 14, 2002 (gmt 0)
newpost thread iconAltaVista Spider?
  trek17.sv.av.com
3HyperGeek 4:27 pm on May 14, 2002 (gmt 0)
newpost thread iconUnknown - Lots of activity
  66.28.250.176
3HyperGeek 4:02 pm on May 14, 2002 (gmt 0)
newpost thread iconTwo mystery spiders
  Heavy crawlers
15Everyman 5:09 am on May 14, 2002 (gmt 0)
newpost thread iconNeed help ID ing a spider
 
2RobertS 12:35 pm on May 13, 2002 (gmt 0)
newpost thread iconWhat is Dsurf?
 
6Brett_Tabke 9:41 am on May 12, 2002 (gmt 0)
newpost thread iconGozilla/4.0
  Just looking for formmail.pl
7bobriggs 9:30 am on May 12, 2002 (gmt 0)
newpost thread iconUniverserseBot/1.0 from 209.17.183.249
  NSlookup says thats gw.activestate.com
2mbauser2 11:21 pm on May 11, 2002 (gmt 0)
newpost thread iconSlySearch/1.0, why is this on my logs?
  why would this be on my logs
23PinkPanther 12:26 pm on May 11, 2002 (gmt 0)
newpost thread iconDIA-Test?
 
4keyplyr 6:25 pm on May 10, 2002 (gmt 0)
Displaying Page 187 of 218<< Back 1 ... 177 178 179 180 181 182 183 184 185 186 [ 187 ] 188 189 190 191 192 193 194 195 196 ... 218 Next >>
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved