Forum Moderators: goodroi
I want to disallow robots from three pages that 'live' in a sub folder called 'pages' I have set up the robots.txt file as follows-
User-agent: *
Disallow: /pages/paypalcanceled.htm
Disallow: /pages/paypalconfirmed.htm
Disallow: /pages/booking form.htm
Is this correct? I do not want to disallow all files in 'pages' only the listed ones.
Also I have saved the robots.txt file to the root directory as a txt. file is this correct?
Thanks for any advice
pjuk
The code posted above may not work as expected, because the Standard for Robots Exclusion [robotstxt.org] specifies that a blank line is to be interpreted as an end-of-record indicator. Delete the blank line, and use:
User-agent: *
Disallow: /pages/paypalcanceled.htm
Disallow: /pages/paypalconfirmed.htm
Disallow: /pages/booking form.htm
Jim