Forum Moderators: phranque
The main reasons for this is...
. I want users to arrive on the pages of the standard website.
. I don’t want a penalty for duplicate content.
Anyone have any better ideas?
You have the right idea on the robots.txt bot exclusion. If you have ssi or cloaking available, I might even go further and not put the link to the directory on the page when the bots visit.