homepage Welcome to WebmasterWorld Guest from 54.226.0.225
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification

Search Engine Spider and User Agent Identification

Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL Forum Options: Reset Last Read Pointers

PreModerated forum to id search engine spiders

 
new Subject Messages Started by Last Message
newpost thread iconWin32; WinHttp.WinHttpRequest.5
  67.113.225.66
4wilderness 12:39 am on May 25, 2002 (gmt 0)
newpost thread iconBlock and Complain success
  Sometimes they do clean up their act
5jdMorgan 5:31 am on May 24, 2002 (gmt 0)
newpost thread iconInteresting spiders from AV & Mercator
 
17Air 3:47 pm on May 23, 2002 (gmt 0)
newpost thread iconCruizer IV
  Does anyone know it
3olwen 8:38 am on May 22, 2002 (gmt 0)
newpost thread iconWhat is this user agent?
  lwp-request/2.01
3korkus2000 9:15 pm on May 21, 2002 (gmt 0)
newpost thread iconZealbot
  LookSmart Spider
2Key_Master 6:43 am on May 21, 2002 (gmt 0)
newpost thread iconHow do I disallow spiders?
  I want to know what code to put on a page to not allow spiders to crawl.
2Mitch 5:03 pm on May 20, 2002 (gmt 0)
newpost thread iconanother one to deny
 
7littleman 5:31 am on May 20, 2002 (gmt 0)
newpost thread iconTcl http client package 1.0
  What is this?
4misosoph 11:22 am on May 19, 2002 (gmt 0)
newpost thread iconAnyone Seen This INK bot
 
3dstanovic 11:01 am on May 19, 2002 (gmt 0)
newpost thread iconAsk Jeeves spider?
 
2korkus2000 1:30 pm on May 18, 2002 (gmt 0)
newpost thread iconAnyone seen this Ask Jeeves spider?
  Ask Jeeves spider - why?
6IanTurner 1:55 am on May 18, 2002 (gmt 0)
newpost thread iconCoding a spider
  How to give yourself an agent name
6chris_f 6:53 pm on May 17, 2002 (gmt 0)
newpost thread iconInktomi
  New C class or did I forget to add this one again?
2msgraph 4:47 pm on May 17, 2002 (gmt 0)
newpost thread iconwebcollage variation
  collage.cgi/1.82
4misosoph 2:09 pm on May 17, 2002 (gmt 0)
newpost thread iconUnknown Crawler
  193.95.83.130
9chris_f 8:07 am on May 15, 2002 (gmt 0)
newpost thread iconspider-loh-tc063.proxy.aol.com
  What Is This
5Funfone 8:44 pm on May 14, 2002 (gmt 0)
newpost thread iconAltaVista Spider?
  trek17.sv.av.com
3HyperGeek 4:27 pm on May 14, 2002 (gmt 0)
newpost thread iconUnknown - Lots of activity
  66.28.250.176
3HyperGeek 4:02 pm on May 14, 2002 (gmt 0)
newpost thread iconTwo mystery spiders
  Heavy crawlers
15Everyman 5:09 am on May 14, 2002 (gmt 0)
newpost thread iconNeed help ID ing a spider
 
2RobertS 12:35 pm on May 13, 2002 (gmt 0)
newpost thread iconWhat is Dsurf?
 
6Brett_Tabke 9:41 am on May 12, 2002 (gmt 0)
newpost thread iconGozilla/4.0
  Just looking for formmail.pl
7bobriggs 9:30 am on May 12, 2002 (gmt 0)
newpost thread iconUniverserseBot/1.0 from 209.17.183.249
  NSlookup says thats gw.activestate.com
2mbauser2 11:21 pm on May 11, 2002 (gmt 0)
newpost thread iconSlySearch/1.0, why is this on my logs?
  why would this be on my logs
23PinkPanther 12:26 pm on May 11, 2002 (gmt 0)
Displaying Page 184 of 215<< Back 1 ... 174 175 176 177 178 179 180 181 182 183 [ 184 ] 185 186 187 188 189 190 191 192 193 ... 215 Next >>
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved