| how to prevent directory from being indexed?
|
dan_popescu
msg:703141 | 9:57 pm on Dec 2, 2002 (gmt 0) |
I just made a robots.txt file for my site. I've included a few files that I don't want indexed. How do I include an entire directory in robots.txt? I know the syntax for files is "disallow: file.htm". Is the syntax for an entire directory (and its content) the same: "disallow: directory"? Thanks
Dan
|
Quinn
msg:703142 | 9:59 pm on Dec 2, 2002 (gmt 0) |
Disallow: /directory/
|
Quinn
msg:703143 | 9:59 pm on Dec 2, 2002 (gmt 0) |
and this should disallow specified bots from everything contained in that folder...
|
dan_popescu
msg:703144 | 10:04 pm on Dec 2, 2002 (gmt 0) |
Thanks Quinn
|
|
|
