Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: goodroi
However, the directories are generated dynamically from time to time as modifications to the CMS are made. The only thing they'll have in common is the first eight letters of the directory name. After those first eight letters will be maybe eight or ten random numbers. (Bad for SEO, I know, but it's a third-party script...)
Is there any way to simply exclude those directories, categorically, using robots.txt? Remember I don't know what they'll be in the future, so I need to exclude them based only on the first eight characters of the directory name. Can that be done, and if so, how?
Also, given that these pages are already indexed, is there a safe way to remove them without affecting the rankings of my other pages?
Then you can ban (in robots.txt) access to somesite.com/directory/dynamic/
(Assuming your directory page uses an HTML redirect and not 301/302?)