Welcome to WebmasterWorld Guest from 23.22.220.37

Forum Moderators: open

Message Too Old, No Replies

how to prevent directory from being indexed?

     
9:57 pm on Dec 2, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 24, 2002
posts:179
votes: 0


I just made a robots.txt file for my site. I've included a few files that I don't want indexed. How do I include an entire directory in robots.txt? I know the syntax for files is "disallow: file.htm". Is the syntax for an entire directory (and its content) the same: "disallow: directory"?

Thanks
Dan

9:59 pm on Dec 2, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 26, 2002
posts:402
votes: 0


Disallow: /directory/
9:59 pm on Dec 2, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 26, 2002
posts:402
votes: 0


and this should disallow specified bots from everything contained in that folder...
10:04 pm on Dec 2, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 24, 2002
posts:179
votes: 0


Thanks Quinn
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members