Forum Moderators: goodroi

Message Too Old, No Replies

newbie to robots.txt, want to exclude /images/

excluding some image folders but not others

         

franco81

1:43 am on Sep 30, 2004 (gmt 0)



I am creating a robots.txt file for a site which has an /images/ folder in the root directory which I want to Disallow with:

Disallow: /images/

But there are a number of folders in the root folder which have their own /images/ folders which I don't want to exclude, will the above line exclude them?

any assistance much appreciated.

jnmconsulting

3:29 am on Sep 30, 2004 (gmt 0)

10+ Year Member



If you use
/images/ then the bot will not crawl the images folder in the root of the web only. to have it not crawl the image folder in the directory "test" the you would put

/test/images/

Hope that helps, do a search on robots.txt and you will find examples and plenty of info on how to use it.