Forum Moderators: goodroi

Message Too Old, No Replies

Duplicate Content in Robots.txt file

         

palmpal

8:26 pm on Jan 29, 2006 (gmt 0)

10+ Year Member



Hi All,

For my robots.txt file I have disallows listed like this one multiple times with different directories and file names.

Should I just have the "User-agent: *" line one time with multiple disallows?

User-agent: *
Disallow: /images

Thanks

Dijkgraaf

12:17 am on Jan 30, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes, each user-agent should appear only once, and then you can have multiple disallow's followed by a blank line.