Welcome to WebmasterWorld Guest from 220.127.116.11
PreModerated forum to id search engine spiders
|Thread Subject||Messages||Started by||Last Message|
|Legit Spider Usage?
Are there ways I can use spiders that don't cause problems for site owners?
|2||OpeRabbit||8:40 pm May 29, 2002|
|Identifying These IP's
Totally Skewed my Log Report
|16||JanCounselor||6:23 pm May 29, 2002|
Spam spider gone mad
|2||estaquieto||2:57 pm May 28, 2002|
|2||dcheney||9:15 am May 25, 2002|
|4||wilderness||12:39 am May 25, 2002|
|Block and Complain success
Sometimes they do clean up their act
|5||jdMorgan||5:31 am May 24, 2002|
|Interesting spiders from AV & Mercator
||17||Air||3:47 pm May 23, 2002|
Does anyone know it
|3||olwen||8:38 am May 22, 2002|
|What is this user agent?
|3||korkus2000||9:15 pm May 21, 2002|
|2||Key_Master||6:43 am May 21, 2002|
|How do I disallow spiders?
I want to know what code to put on a page to not allow spiders to crawl.
|2||Mitch||5:03 pm May 20, 2002|
|another one to deny
||7||littleman||5:31 am May 20, 2002|
|Tcl http client package 1.0
What is this?
|4||misosoph||11:22 am May 19, 2002|
|Anyone Seen This INK bot
||3||dstanovic||11:01 am May 19, 2002|
|Ask Jeeves spider?
||2||korkus2000||1:30 pm May 18, 2002|
|Anyone seen this Ask Jeeves spider?
Ask Jeeves spider - why?
|6||IanTurner||1:55 am May 18, 2002|
|Coding a spider
How to give yourself an agent name
|6||chris_f||6:53 pm May 17, 2002|
New C class or did I forget to add this one again?
|2||msgraph||4:47 pm May 17, 2002|
|4||misosoph||2:09 pm May 17, 2002|
|9||chris_f||8:07 am May 15, 2002|
What Is This
|5||Funfone||8:44 pm May 14, 2002|
|3||HyperGeek||4:27 pm May 14, 2002|
|Unknown - Lots of activity
|3||HyperGeek||4:02 pm May 14, 2002|
|Two mystery spiders
|15||Everyman||5:09 am May 14, 2002|
|Need help ID ing a spider
||2||RobertS||12:35 pm May 13, 2002|