Forum Moderators: phranque
I have had good deep crawls from robots into hundreds of my clients' sites and have never seen any results of password protected material on the web. I have, however, seen one of their user lists posted! I fixed that with a robots.txt, and if you are concerned about protecting your password protected side, I would too!
The only way a bot could accesss password protected documents is if it followed a path which included this password. These links are obviously a huge security hole and IMO should never be used, however occasionally a user will do this to avoid signing-in each time.
If you find this type of access in your logs, I would immediately discontinue access for the username/password, then contact the user and explain your security concerns. There may also be methods of restricting the use of these links using mod_rewrite if you are on Apache.
That being said, other types of optimizations, the kind that can improve performance can be a help to your users who are using this password protected portion.
