Forum Moderators: goodroi

Message Too Old, No Replies

excluding cgi-bin

is this right

         

rbacal

2:39 am on Nov 10, 2004 (gmt 0)



I know this is way basic but I need to be absolutely sure I get it right. I don't want any robots to spider my cgi-bin directory. Is the following correct, if placed in the directory where my web pages live (httpdocs)?

User-agent: *
Disallow: /cgi-bin/

ncw164x

2:59 am on Nov 10, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes it is the correct format

A robots.txt tutorial and validator
[searchengineworld.com...]
[searchengineworld.com...]