homepage Welcome to WebmasterWorld Guest from 54.204.141.129
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Marketing and Biz Dev / Deprecated - Search Engine Submission
Forum Library, Charter, Moderator: open

Deprecated - Search Engine Submission Forum

    
how to prevent directory from being indexed?
dan_popescu




msg:703141
 9:57 pm on Dec 2, 2002 (gmt 0)

I just made a robots.txt file for my site. I've included a few files that I don't want indexed. How do I include an entire directory in robots.txt? I know the syntax for files is "disallow: file.htm". Is the syntax for an entire directory (and its content) the same: "disallow: directory"?

Thanks
Dan

 

Quinn




msg:703142
 9:59 pm on Dec 2, 2002 (gmt 0)

Disallow: /directory/

Quinn




msg:703143
 9:59 pm on Dec 2, 2002 (gmt 0)

and this should disallow specified bots from everything contained in that folder...

dan_popescu




msg:703144
 10:04 pm on Dec 2, 2002 (gmt 0)

Thanks Quinn

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / Deprecated - Search Engine Submission
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved