Welcome to WebmasterWorld Guest from 54.226.246.160

Forum Moderators: goodroi

Message Too Old, No Replies

excluding cgi-bin

is this right

     

rbacal

2:39 am on Nov 10, 2004 (gmt 0)



I know this is way basic but I need to be absolutely sure I get it right. I don't want any robots to spider my cgi-bin directory. Is the following correct, if placed in the directory where my web pages live (httpdocs)?

User-agent: *
Disallow: /cgi-bin/

ncw164x

2:59 am on Nov 10, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes it is the correct format

A robots.txt tutorial and validator
[searchengineworld.com...]
[searchengineworld.com...]

 

Featured Threads

Hot Threads This Week

Hot Threads This Month