Forum Moderators: open
And another issue, I use a custom in-house CMS to run all the content. Should I put it so googlebot isn't allowed to visit those directories? I am afraid that someone might look in the robots.txt and see the directory for the admin login and somehow try to hack it. I will never have links pointing to that directory so is it better to just assume no one can get to it then and neither will google? What would you guys recommend under this situation?
Thanks!
</bump>
There is usually no need to include un-linked, password-protected directories in robots.txt, for the reasons you cite.
Jim